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ABSTRACT 

In modern society, lifelong learning by the 
physician is essential. This study was conceived to: (1) define the 

educational needs ot practitioners in the Pacific Northwest, (2) 
assess the resources available to meet these needs, (3) determine 
what educational programs are needed, (4) develop evaluation methods 
tor these programs, (5) identify physician participation factors, (6) 
develop evaluation techniques tor clinical communication systems, (7) 
evaluate recent continuing education programs, and (8) develop a 
comprehensive plan for a continuing education center. The report 
describes in detail the techniques used to accomplish these 
objectives, with questionnaires and survey results contained in the 
appendixes. Ideally, medical education should continue in a lifelong 
pattern developed during undergraduate study. With improvements in 
program content, promotional efforts, information networks, and 
program offerings in individual hospitals, continuing education can 
be made an integral part of physicians* careers. (BH) 
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In September , 1964, the* Project Director, William 0. Robertson, M.Ih, 
Assoc [a to lean, outlined in a lf.-ttci ‘ the Doan of Medicine at the 
University of Washington the* potentials f ai. Of . ice of Research in 
Medical Education, w >ic one created at the University of Washington 
(Appendix A). Received favorably, the next step involved seeking finan- 
cial support. At approximately the same time, Dr, John Rein joined the 
University of Washington as Director of Continuing Medical Education; 
his acceptance of this position was the initial step in elevating the 
function of continuing nedical education to a status equal to that of 
educating nodical students and house officers. In his new capacity, 

Dr, Lein also exprcoscJ interest in the benefits to be derived from an 
Office* of Research in Medical Education — especially in the continuing 
education field. Such war, seen as particularly relevant in formulating 
programs in his relatively new venture. 

When these aspirations v;ere shared with representatives of the 
Public Health Service and its then Division of Community Affairs (Miss 
Cecilia C. Conrath) — particularly in view of the unique geographic, 
educational and historical facets of our situation-interest was ex- 
pressed in possibly developing a collaborative program by contract. 

Over several months, the challenges of a contractual relationship were 
worked out. Of significant import i«; the fact that, simultaneously , 
details of the Regional Medical Program were also being worked out 
nationally and locally and, from the beginning, possibilities of 
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overlap were recognised. Thus, by .July, 1966, the contract was signed 



with nine specified objectives. It should he note l that the agreed- 
upon program also served as an Initial step within the University of 
Washington School of Medicine in developing its Office of Research in 
M ed i c a 1 Ed u c u t i o n 

Recruitment problems precluded the appointment of a qualified 
Program Director for more than one year. Fortunately, in July, 1967, 
Dr, Charles W, l)o>' ;r agreed to join the faculty of the University of 
Washington both as Program Director and as Director of the Office of 
- Research in Medical Education. While several activities had begun prio 
to his arrival, only subsequently did full pursuit of objectives begin. 
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The development of r. study such ns this involved several indivi- 
duals. We sought advice from and would express appreciation to many 
people , including Miss Cecilia C, Conr.ith, who assisted in drawing, up 
the original contract; Dr, Alan S, Kaplan and Dr. John Lscovitz for 
their encouragement; and two Project Officers, Mr. Norman E, Tucker 
and Mr, Norman E, Crotiquist, for their valuable guidance as the con- 
tractual objectives were pursued. 

Dr, LeVerne S, Collet and Mr, Ronall L. Hanberg, research assist- 
ants, provided invaluable assistance in areas of statistics and research 
design, Mr. Darrell Hull and Mr, llajgh Fox contributed of their t.mo 
arte expertise in the compilation and analysis of much of the data, 

Mrs, Lurie Praciit and Mrs, Veit a Denson have been most patient, under- 
standing, and cooperative iu performing secretarial and other tasks 
essential in such a study. 

Without tlie cooperation of the Vachington State Medical Education 
and Research Foundation staff, the Washington/Alaska Regional Medical 
Program staff, the Division of Continuing Medical Heine? Lion at the 
Univcisity of Washington, and a host of physicians who willingly parti- 
cipated in these studies, this study could not have been completed, To 
all of these individuals, we art* extremely grateful, 

W.P,R, 

C.W,D. 
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SECTION I 



Tin: mBjxi 

This decade has seen continuing medical education along with under- 
graduate curriculum revi r> ion , ai'missi oas of minority student:;, arguments 
about relevance, f'he "mediums of the message" concept — these and other 
issues vie for notoriety on the medical education scene. Certainly, 
a host of factors have been responsible for the resultant "unrest* 1 ; a 
few seen particularly contributory in spurring on continuing medical 
education, Highlights include: 

1 . The scientif i c and techno logic explosion. While b:' ^medical 
historians can substantiate the point that no actual change 
in the rate of growth has occurred since the 170L) r s, in abso- 
lute numbers the products and development s have become vir- 
tually overwhelming , W v cn one considers that 30 to 90 per 
cent of all the scientists that have ever lived are 3 wing 
today--and the number can be expected to double within 15 

to 20 years — and that 80 to 90 per cent of today’s drugs 
were unknown in 1940, the basis of the current dilemma 
becomes obvious. 

2. L commi tment to specialisation iji medicine , Starting at 
the turn of the century and reaching full steam immediately 
after World War 11, specialization night be said to have 
run anuck. As a consequence, today’s practitioner both 
expects and is expected to be far better informed in speci- 
fic areas than was his predecessor of a generation ago. 
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3 . T he comma n \ ca t 1cm r o' vo .Hi t ion . Ob v I < hi a 1 > , with the advent 
of the radio, the telephone, television, and computers, 
information transmittal techniques h ve taken a quantum 
jump comparable to that initiated by tho arrival of the 
printing press 300 years ago. 

4. The transpor t a tlon explosion* With the at rival of the 
modern automobile roadway network nud jot ail planes, the 
potential for isolationism and reaction it ics would seen to 
be lost. 

5. A violent d isrupt ion of soci al philosophy . Now the public 
and its politicians as well as the profession would seem 
to subscribe to the theme that "health C3rc is a right of 
the entire population. 11 As a consequence, costs of health 
care are receiving increased scrutiny; cries to enhance 
th«' efficiency of the health care system echo throughout 
Congress — ard over the United States. All make new and 
different demands as yesterday’s health professional 
assumes a new image tomorrow. 

6 . The reincarna tion of hum anism . In the years immediately 
following World War II, science becatae deity. However, 

tlie past three years has seen a burgeoning of considerations 
of philosophy, ethics, moralities, etc., on campuses and 
among the population. The reaction has redirected, to 
some extent, the image of what constitutes acceptable con- 
tinuing education endeavors in the eyes of many professionals. 
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7. Exploitation of the Hawtho rne effect, Identified during 
World War II, the concept of Inducing change for change's 
sake and its resultant improved behavior has gained wider 
an l vldw' recognition. The Import of this Ingredient in 
the personal as veil as professional life of the physician 
is being recognized with increased frequency. 

The summation of these and other factors makes more than plausible 
the need for physicians to acquire additional knowledge, skills and 
attitudes after leaving the formal education years. Put differently, 
lifelong learning becomes critical. 

In centuries gone by, the individual physician to a large extent 
not only was responsible for, but actually tended to, the conduct of 
his individual updating. Soon such a process was discovered not to be 
efficient enough to handle the load, but attention to alternatives 
awaited the Middle Ages. On the one hand, social organizations such 
as professional societies and, eventually, hospital staffs were formed 
to assist physicians with continued learning. On the ether lin'd, com- 
munication technology (books, then journals) was employed to respond 
to the challenge. Today's new technologies offer still more possible 
responses; today's complex organizations would sce.n to do likewise. 

But burgeoning load and specialization of needs, as well as an over- 
hvoadenlng concept of the physician’s role, seem to negate the combined 
effo f organizations and technology in catching up with the ilti- 

iriatc go*, i. 

At tire tine this contract was being proposed, individual and 
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organizational approaches to continuing medical education were mush- 
rooming in the Pacific Northwest. Each physician and each organization 
seemed to be doing more and nore--vith virtually no regard for what was 
being dona by others* While all agreed to the general need, neither 
individuals nor organizations had begun to undertake any critical anal- 
ysis of the. specifics involved. Nor had they capitalized on interdl- 
gitating and exploiting the existent communication technologies. The 
individuals concerned with th*; preparation of the contract and their 
sponsoring organizations agreed to the value of examining the issues 
carefully and recommending a continuing medical education plan which 
would b p . applicable to the Pacific Northwest. Thus, the jumping-off 
point was reached. 
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SCOP!- OF WORK 
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1. Define the educational nejds of practitioners in the Pacific 
Kor t hue at • 

2. Assess the resources available for meeting these needs, 

3, Determine the types of educational programs which should be devel- 
oped to meet the needs of the physicians, 

4, Develop a system for evaluating the effectiveness of these 
programs, 

3. Select two continuing education programs as the model for investi- 
gating why some physicians chose to participate in these programs 
and why some chose not to participate. 

6. Develop and field test evaluation procedures of clinical communi- 
cation systems, e.g. , hospital patient discharge summaries cent to 
local medical doctors, telephone conversations, and/or correspond- 
ence between hospital and staff and patient referring physicians, 
etc, A major criterion of the evaluation shall be the effect upon 
patient care, 

7. Evaluate the recently initiated seminar program conducted by repre- 
sentatives from the Schools of Medicine, Architecture, Education 
and Communication Arts, for those faculty members participating in 
continuing medical education programs. The purpose of this seminar 
is to improve understanding of the learning processes and to imp, 
teaching skills, 

8. Select two additional continuing education activities conducted 
by the School of Medicine for evaluation purposes. Activities to 
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be selected ray include: relevance of medical student appraisal 

programs for practitioner education; relevance of techniques cur- 
rently employed to assess needs of residency training programs; and 
methods to improve the quality and impart of nodical journals# 

9. Develop a comprehensive plan from the activities (1 through 5 
above) for a continuing education c :ntor for physicians. This 
plan shall: 

a. Determine the functions to be carried out by the center. 

b. Set forth working procedures to be followed in establish- 
ing the center. 

c. Select specific areas of study and priorities of work to 
be undertaken by the center. 

d. Recommend a detailed scheduling of tasks to be undertaken 
for implementation of this plan. 

Specific study areas to bo examined in Items 1 through 5 above were 
selected and agreed upon in advance by the Contractor and Project 
Officer . 

P>ecause a considerable length of time had elapsed between the 
initial dete of this Contract and the employment of staff to pursue 
the contract objectives, adjustments needed to be made regarding spe- 
cific details of each objective. A review of the objectives was made 
in collaboration with the T^.gran Director and the Project Director, 

As a result of tills review, it was decided that no further work need 
be done on Objective 1 since these activities were now history, hike- 
wise, specific activities were agreed upon for each of the other 
object ives . 
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An attempt h..s been made co describe the methods used and t he 
problems encountered in the study. Section II contains a discussion of 
the method used to achieve Objectives 1 through 9. In addition to the 
methods, excerpts of some of the pertinent findings are included where 
it is deemed appropriate . A discussion of the problems encountered in 
pursuit of this contract and recommendations based on the findings from 
the various studies undertaken arc to be found in Section III# 

In the Appendix are more comprehensive records of studies under- 
taken in fulfillment of the various objectives. IncJuded are papers 
which have been prepared under the aegis of this Contract, some of 
which K rve been published; others have been submitted for publication. 
The survey of Washington State physicians’ attitudes toward continuing 
medical education is also included in the Appendix due to its relevance 
to the pursuit of the objectives of this Contract and the cooperative 
efforts of the Program director, Washington/Alaska Regional Medical 
Program, and the Washington State Medical Association. 



SECTI ON Jl 

This section is a review of the methods followed in each of the 
specific phases (Objectives 1 through 9) of the study. Excerpts from 
some of tile findings are included where such seems to be pertinent. 
More detailed reports can be found in the appeiuiix and in previous 
quarterly progress reports. 
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OBJECTIVE 1: Define the educational needs of practitioners of the. 

Pacific Northwest . 

Two general approaches to this issue were employed; the fjjr.s_t 
involved asking the physicians for their personal perceptions of needs 
and the second involved drawing on the existent data accumulated by the 
epidemiologists regarding the existence of health problems, accumulated 
by health educators in terns of professional learning needs and extra- 
polated from a review of recent scientific advances which had trans- 
pired subsequent to the graduation of the vast majority of the area f s 
physicians « 

Since the advent of continuing education activities (brief courses) 
with the University of Washington in the 1950's, opinions had been 
extracted from attendees regarding their appraisals of the courses 
offered and their suggestions of topics for additional courses. Sum-* 
marization of these opinions provided some guidelines in the evolve- 
ment of a questionnaire to be completed by physicians in this area. 
Because of understandable restrictions on the use of mail questionnaires, 
it was deemed virtually impossible to conduct a questionnaire survey of 
the physicians under the terms of the contract within the desired Lime 
interval. Fortunately, however, the Washington State Medical Education 
and Research Koundat ion--a branch of the State Medical Association--had 
simul tanen jsly decided to poll the State physicians regarding their per- 
ception of continuing education reeds. Funds for thin survey were pro- 
vided by the Washington/Alaska Regional Medical Program and, as noted 
in th ir final report, they received consultative advice from the 
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Project Director nr; veil ns the Program Director. Our participation in 
tins survey and use of subsequent data to contribute to our achievement 
of Objective 1 was approved by the then Project Officer, Miss Cecilia 
Conrath. 

At the same time, the Project Director was fortunate in being invi- 
ted co p, :ticip. te with a group of physicians in the South Seattle area 
as they discussed among themselves their "educational needs" and then 
evolved a program intended to respond to their stated needs* Interest- 
ingly, this pilot program proved so successful it is continued today 
and is now self-supporting. 

In the spring of 1967, the School of Medicine and the Washington 
State Medical Association agreed that their annual faculty- community 
physicians' retreat ought be focused on the issues of continuing medical 
education. Chaired by the Project Director, the program was preceded 
by a sampling of the attendees* "continuing education behaviors" for 
the previous year. Admittedly a selected sample, it nevertheless offers 
postulated advantages of serving as a leadership cadre which conceivably 
may be emulated by the "masses." Somewhat surprisingly, five possible 
options available as "ny biggest problem in continuing education" 
received almost equal attention. This group saw "learning to apply 
what is new," "finding out what is new," "corroborating what I think I 
know," "applying v!iat I already know," and "discording what is outmoded" 
all as important, A number of other findings were of interest, e,g., 
the favorable reception to "circuit-riding courses," the desire for 
television, the use and non-use of nodical journals, etc, 
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One point :1s of particular significance. When asked to specify 
three "significant new medical advances" and the source from which they 
were obtained, the sample of 5 2 respondents cited virtually totally inde- 
pendent items of information* Thus, each individual seemed to see dif- 
ferent items as being significant to hlcu Of 100 such items reviewed, 

A6 were purported to hive been gathered from medical Journals, 20 from 
hospital staff meetings, 27 from attendance at courses or national r;eet- 
ings, A from consultants and 1 each from "Audio Digest," a telephone 
conversation and a popular magazine. The individual i ty and diversity 
of topics cited and the variety of sources alluded to would appear to 
confirm some of the implications of the attached editorial, "Sock It 
To ds<" 

Also in the development of a "definition of need," the Regional 
Medical Program employed Mr. Robert Neth of the University of Kansas, 
Department of Continuing Education, to survey the state of Alaska with 
regard to continuing education needs for physicians. Results of this 
survey were made available to us; consequently, no repeat »survey was 
undertaken, since ve chose to avoid duplication of efforts. 

With reference to data gleaned from the existent literature, it 
should be pointed out that in 1953 the "Washington Sickness Survey" war, 
completed with appraisal of reasons for 73,188 patient visits to physi- 
cians throughout the state. At the time of its publication as a mono- 
graph as well as an abbreviated form in NortJ^wost Medicine, this was a 
first of its kind. While no subsequent survey of total illness burden 
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has been conducted in our total region, they were available from elsc- 
v/here. Simultaneously, the Regional Medical Program initiated a "patient 
origin study" intended to document the hospital illness burden as veil 
as the geographic source of patients hospitalized throughout our region, 
While these data care not yet complete, no real reason exists to believe 
that the distribution of various illnesses is likely to be dramatically 
different in our region as compared to other regions in the United 
States — with the usual exceptions well-known to health officers, such 
as histoplasmosis, tuberculosis, etc. 

Two brief reviews were conducted concerning the adequacy of treat- 
ment of two specific situations— *onc in the hospital, and one in the 
community. The underlying presumption was that any deficit treatment 
applied could conceivably be construed as pointing to educational need. 
The results were tabulated and reviewed in terms of adequacy. 

As noted under Objective a chart review was conducted in 
Yakima regarding the treatment of stroke under the fiscal auspices of 
the Regional Medical Program but with consultative direction from the 
Program Dir ctor. Simultaneously, a four-part television scries was 

produced to outline and then depict conventional and acceptable approa- 

t • 

chcs to management , The post-program review of charts isjbotng conduc- 
ted^-rbgain under the aegis of the Regional Medical Program- -this summer. 
In part, the deficits uncovered Initially did help to determine the 
educational program which followed, 

V/e wore also fortunate In being able to capitalize on the arrival 
of a "medical breakthrough" during the existence of the contract. In 
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May, 1%8, RHO GAM — immune* Pvh globul i n--becane commercially available. 
Injected into Kh-negative ir« others within 72 hours after their delivery 
of an Rh-posii jlVo. offspring, this precludes the subsequent development 
of a hypersensitive state in the mother — and therefore excludes the 
development of problems with the next fetus. Since, in our county, all 
prenatal bloods are typed in a single station and all U1I0 GAM is dis- 
tributed from the same single station, we were able to compute appro- 
priate "use rates* " At the end of the first three months of commercial 
availability, at least S3 per cent of the 167 patients at risk in the 
King County area did receive the appropriate treatment; another 8 per 
cent of patients are known to have declined to accept the treatment for 
either religious or economic reasons. Thus the information had been 
disseminated extremely rapidly and widely into the practicing community* 
In this instance, it would seem that no "need" existed for additional 
“education," We went further and conducted a telephone inquiry into a 
sample of 24 physicians seeking information as to their own sources of 
information and, what we considered to be more significant, their under- 
standing of the "physiologic basis" involved. Interestingly, 22 out of 
24 respondent physicians were judged as having remarkably complete 
understanding of the pathophysiology and immunopharnncology involved, 
Finally, as noted under Objective 2, a Council of Community Coor- 
dinators has been mobilized to still further carry on the identifica- 



tion of individual and collective needs. 
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OUJKCTIVj; 2: assess the resources avail;. .jIu for meeting these needs. 

Formal and informal methods were used to assess the resources avail, 
able for reccing continuing educational needs* In approaching the 
assessment of resources within this state, answers to the following 
questions seemed pertinent* 

1. To what degree do the various medical groups now provide 
continuing medical education programs? 

2. Who arc the key per sonnel throughout the state who have 
an interest, and who are actively involved, in supporting 

' continuing medical education efforts? 

3. What are the sources of financial support currently being 
used ? 

4. What, if any, instructional media are readily accessible 
for those sponsoring continuing medical education pro- 
grams throughout the state? 

In an attempt to find answers to the above questions, the Division 
of Continuing Medical education at the University of Washington (in 
cooperation with the Office of Research in Medical Education) distri- 
buted questionnaires to four medical groups within the state tint were 
potential resources. This questionnaire was sent to appropriate staff 
persons in the 126 hospitals, 27 medical societies, 51 learned socie- 
ties, and to the 13 county health officers throughout the state. A 
follow-up letter was sent to those not responding on the first mailing. 
Table 1 shows the percentage of returns which range from 56 per cent 
for the county medical societies to SO per cent for the learned societie 
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Responses to four questions are presented in Table 2. It must be 
kept in mind tint those findings are based on partial returns. Vifty- 
nlnc per cent of the. hospitals have formal continuing nodical education 
programs while only 3 per cent of the county health officers sponsor 
such programs, Clearly, the greatest resource nr-ung those responding 
in these four groups for continuing medical education is the hospital, 
However, 41 per cent of the hospitals responding did not have any 
organized continuing medical education programs. T1 is leaves a vast 
resource that is only partially being utilized, to say nothing of the 
need for improving existing programs. 

About half of the hospitals indicated that they have a staff mem- 
ber defoliated to coordinate continuing medical education programs. 

Very few of the other groups have such a person so designated. The 
fact that the hospitals do have a person in this position may account, 
in part, for the higher percentage of continuing nodical education 
programs. The appointment of a staff member to assume th*.s responsi- 
bility would seem to be imperative if organized continuing nodical 
education programs are to become an integral part of the hospital's 
educational program. At the tine of this inquiry, the hospitals and 
county health officers wore fairly veil -informed as to the opj-nrl uni- 
ties under the Washing ton/ Air ska Regional Medical. Program; however, 
only about one -fourth of the county medical .societies and learned 
societies were sc. inlorted. 

The preponderance of continuing nodical education programs spon- 
sored by these f mr groups was directed toward the practicing phy* i« iana, 
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Very few of them were designed for allied health personnel. Kith the 
exception of 21 hospiLals who indicated they either have more than one 
program per week or less than one per week for allied health personnel , 
continuing, education programs for this group of health professionals 
are practically non-existent* If those who responded to this inquiry 
do actually represent the medical community's' continuing education 
efforts for the allied health personnel, then one might well conclude 
that a great need docs exist In this area. On the other hand, there 
may be other institutions or mechanisms whereby these health profes- 
sionals can keep updated. This is a question for future investigations. 
Table 4 presents the proportion of continuing medical education 
programs funded by dues, tuition, drug companies, or not funded at all. 
Thirty-one of the hospitals reported that over 75 per cent of their 
programs were not funded by any outside source. For the county medical 
societies, 25 reported that up to one-fourth of their programs were 
funded by the drug companies. Ten of the learned societies indicated 
that over 75 per cent of their programs were funded through dues. 

The various educational media available for continuing medical 
education programs in these four organizations are reported in Table 5, 
Only those indicating they had forma) continuing medical education pro- 
grams were asked to respond to this question. The hospitals have by 
far the. most equipment available, of which the most popular are slide 
projectors, movie projectors, and tape recorders. The county medical 
societies' and the county health officers did not appear to have much 
equipment readily available to then. Future continuing nodical 
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education programs which require such rod la as television, movies, two- 
va/ radio, tape, etc,, r.usfc be programmed with these needs in wind. 

The coix.^-nts of respondents from each of those four organizations 
arc listed umSur item numbers 7, 8, and 10 which deal, respectively, 
with ideas about educational programs that are needed in the future, 
unique resources available 3n their area, and other comments or 
suggestions, (a copy of these comments is at tached . ) 

In addition to this more formal approach to enumerating resources, 
informal discussions have gone on between the project staff of this 
Contract and interested persons in the Washington/Alaska Regional 
Medical Program, the Director of Continuing Medical Education at the 
University of Washington, the Executive Secretary of the Wa \ington 
State Medical Education auJ Research Founda tion , and the 20 Community 
Coordinators scattered throughout the state of Washington, During 
1963, a pilot study was conducted in four cities using computer-assisted 
instruction as a resource for continuing medical education. Also, 
through the Project Director, Dr, Will lain 0. Robertson, cooperative 
efforts have been made in making use of two-way radio programs in con- 
junction with sources fie,-) the state of California available to the 
people in the state of Washington as well as the state of Alaska. 

During this two-year contract period, wc have seen a decided 
increase in the use of local television facilities as a media for sup- 
plying continuing medical education programs to physicians throughout 
the state of Washington. The resources of the University of Washington 
medical complex, the Washington State Medical Association, and the 





18 



Waybill to, /Ansha Regional Medical Prop, ran are now available through a 
centra L organizational structure.. Thus, individuals or local commit- 
tees car. request assistance frotj a single source in providing continuing 
educations ■ programs to meet the needs oJf the local cowjruir ity . This 
type of a resource has hitherto been unavailable to the physicians in 
the NorthvosL area. In fact, each of the Community Coordinators has 
recently been supplied with an Autloscan projector, carousel program 
"0 M slide projector > and Fairchild movie projector, 

Iherr- are ?3 community colleges within the state, most of vhich 
hive an adult education program. We arc beginning to use this resource 
more effectively. For example, regular continuing nodical education 
courses developed for one community are now being conducted through 
the local community colleges 1 adult programs. The college supplies 
space and equipment as well as handles the administrative details. 
Increasingly, the community colleges will serve ar a significant role 
in continuing medical education efforts. 







Finally, three additional resources warrant recognition: 

1, The region is served by a regional medical Journal, "North- 
west Medicine," which simultaneously serves as the official 
journal for three stntes--luishington, Oregon, and Idaho, 
(Alaska has its oval state medical journal, "Alaska Medicine, " 
edited by A. Van Hippie, M.0.) "Northwest Medicine" has 

Dr, Herbert Hartley- -a recognized authority in the field of 
medical pub] lent ions--as its editor; he is assisted by a 
region-wide Editorial hoard of which the Project Director 
serves as Chairman, Periodic sampling surveys have been 
and are being conducted among readers regarding this par- 
ticular vehicle; increasing efforts are being made both to 
increase i^s stature as a repository f r prestige scienti- 
fic articles and to increase its usefulness as a local- 
regional medical newspaper covering significant happenings 
on the medical, social, economic, educational, ana psycho- 
logical forefronts, 

2. During the pei iod of this contract, the Pacific Northwest 
Health sciences Library was born. Under the sponsorship 
of the National Library of Medicine, this "facility" is, 

in fact, a regional cooperative arrange lent among chc vari- 
ous health sciences libraries in the Pacific Northwest , 
hospital libraries, medical society libraries, public 
libraries and enthusiastic librarians. Focusing on reprint 
services, b ibliogi aphic searches, and MLDLAKS searches, 
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tills organization is providing an invaluable service to 
tlie area's health professionals responding, for example, 
to all except 14 of the first 32GD requests generated to 
it in its initial three nontlip af operation. 

3. Also daring this tine interval, as vas noted in tiie intro- 
duction, the Wn shi ngton/Alaska Regional Medical Program 
(U/ARMP) cane into being. One of its primary efforts has 
been in the field of continuing medical education. Working 
closely with the University and professional societies, both 
its Director, Dr. Donald Sparkman, and its staff have been 
extremely cooperative in sharing ideas and data with the 
Project Director and Program Director and have made avail- 
able several of their projects as a "laboratory” for some 
of our investigations. 

The W/AHMP financed the Washington State Medical Association's 
study of the continuin' educational needs of its members. In part as 
an outgrowth of this endeavor, these two organizations Joined with the 
University's Division of Continuing Medical Education in evolving a 
unique, area-w T ide organization. Leadership was provided by Dr. John X, 
Lein, Associate Doan for Continuing Education, Dr. Thomas Shechy, Chair- 
man of the Washington State Medical Association's Committee on Continu- 
ing Education, and Dr. Robert C. Davidson, Associate Director of the 
Regional lfedic.il Program, in forming a "Coainuulty Coordinators' 
Educational Council." Invoking the following premises, 







1 . 



C o n t i n u i n g o tl u c a 1 1 o n 1 s bo t h a n ind i_v i d u 1 f u n c L L co - - 
reading’,, talking;, etc., and an orgaji 1_z a t_i on ;i\ fu r o n - - 
radio, movie, telephone, meetings, television, etc. 
both a re critical, 

2. The individual II. 1). should have an opportunity to put in 
ideas at all stages of continuing education as should 
their organizations — Washington State Medical Association, 
University of Washington, W/AKMP, etc. 

3. A system of continuo us continuing education functions is 
critical as are the continuity cogs in it--i.e., both 
people and organ! /.at ions ought to participate for long 
enough intervals to be optimally effective, 

these three organize Lions agreed as follows: a council ought to be 

created v,*itli broad representation to assn r c c o n t i nu ou s individual in- 
put to participating organizations. At the same time, individuals may 
be invited from this council to par tici pate in the output »ide at the 
local level. Thus the council c ompr ises the "communication focal 
point" between .he "individual" and "organizations" and amo ng the 
organizations. 

The main responsibility of the council is "To serve as a forum 
regarding continuing nc-dical education with involvement of the School 
of Medicine, the Committee on Continuing education of the Washington 
State Medical Association, the Washington State Medical PJucat ! on and 
Research Foundation, and the Wish I pg l on/Ala ska Regional Medical Program 
as well as other individuals and organizations in the region. 
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Fornnlly organized in September, 196b, and activated in December, 
this resource is rapidly becoming; the integral network coordinating con- 
t inning r.v.-d Ir* ;i 1 educational activities throughout the stat e* Interest- 
ingly, it is likely to be emulated in tike Alaska region shortly. (See 
also text rclaLiny to Objective 9.) 
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KESFO'IDENTS OF A SURVEY OF COM! IA'UIKU MED I CAE 
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CONTINUING UrDTCAL l'OUCATION' PNOCEAMS FOE ALLH-!) iiKAU'!! PlIKSONNr.l. 

spo::so:<i:i) oy hospitals, county mtdtcal socinruis, county ucalth 

OKFICi'NS , AND MIA JUT YO SOCJLTIUS IN WASHINGTON, 1967.* 





Hospitals 


County Med ■ 


County Health 


Learned 


Number 




Society 


Officers 


Soc * 


franc 


38 


13 


21 


35 


Very fo;. r 


20 ; 


2 


1 


5 


Less than one per week 


15 


0 


1 


1 


More than one per week 


6 


0 


1 


- 



* Allied health personnel defined as all health personnel except 
physicians and nurses. 
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svmo .vr to: ;.\L mi:iha availauu: for co:;ri:un:;c mcdicai. nuircATiu.; 
i:; hospitals, (jo:;:;rv . ;::dtcat, fociclifs* cclaty ijfat.th 
ovftcfrs, A:Ci) lf \k::k3 sociftus i;, iJasiukwto;;, 15C7,* 



lied ia 


hospitals 


County MeJ . 
Soc . 


County Health 
Off leers 


Con mud 
So c • 


Slide Projectors 


42 


5 


4 


12 


Television 


12 


0 


0 


; 1 


Movie 


38 


5 
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Faitenild Movie 


10 
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Two-way Radio 


2 i 
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0 
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Tape Recorder 


31 


0 


1 

0 





* Only thorio I:iJ J eating they had formal continuing ucdlc.il i ! 
programs veto asked to respond to this it on. 
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commits 



Ho^ji it a 1 jj 

7 . i'du e a t i o r s " l Km ;o 1 1 r ( ■ v r , : 

Specialists to lecture, in n note a ree s on or t hope dies , F.N V , nl.lt rgy. 
Hove carefully selected movies, current films, 

Mora in forna 1 ity--opon rxcli.uif^s s-rsill groups. 

Review courses ir. all specialties — short , 2-3 days or leer. 

More organization locally* 

General floor nursing tcclmlr-ue review; coronary and intensive 
care nursing. 

State-wide program of televised, movie, or Fairchild presentations 
for nurses, technicians. 

New material and information on drugs, toxic reactions to new and 
old drags* 

Need more staff cd a cat ion- '-hard to get cooperation. 

Mortality discussions monthly* 

Wet cHnics utilizing discussants abreast of latest research. 

8. Resources: 

Green River Valley postgraduate nodical education; two courses 
yearly at Green River Community College, Highlinc Junior College. 
Highly- ta 1 en ted nodical population in Spokane. 

Shrine Hospital in Spokane limited to children's orthopedics. 

Occasional unusual cases* 

Central Washington continuing och :ation program. Regional Medical 
Program. 

A central emergency room (HarfcorvU w) . 

Comprehensive RehabiJ itat ior. Center (Good Samaritan, Puyallup). 

Heart Center, Coronary Care Unit* 

10. Other comments or suggestions: 

"Reefed up" programs would attract private physicians in area. 
Organization (Pacific C< inty) does not meet. 

1 would attend all surgical meetings and heart courses. 

Films usually obtained from pharmaceutical firms. 

M t , Linton Hospital in Mctallina Fails has no doctor, 

Staff consists of only two M.D.'s* 

Would like to know about meetings at other hospitals. Wo aLLonl 
as often as possible. 

Staff is so small that continuing education mist be done on a 
personal basis instead of a hospital program. 

Statewide int. gr itec! program provided jointly by the Region) 

Medical Program and University of Wash i ngtoti , and directed by 
you ! 

Current tlv.nwj mint be r,i mued . 

Mar front hly staff meetings Invoke case d i scansions — f Veguont ly ir.-J.pth. 







COMM } r S ( co r* t i m j t J ) 



Co un t-V - "I N L ' ' O 1 S 0 C 1 C t i < ■ S 
7 , Kclu at j on 1 1 Programs : 



More ab.n.: Ifjrif.'y— 1 ess hard facts r social issues In medio ine 
oca i e':p lor Lug. 

CnrrciL tkerapout Lc:> , rcfrerh^r courses in subr >pc ini t Hs , prac- 
tical do .lonstrnt ions of methods of diagnosis. Stop lectures. 
Prorr^’t orships for practicin'; physicians arc great. 

Circ.ii 3 1 rider program should not bo discontinued . 

Cut rent clinical, current research, federal and state legislation 
inj net , 

Almost any topics wo] come-- traveling groups from medical centers 
could give concise Information of general interest. 

Topics: US AD names of drills; liarmonal therophy of menarcho; 

medical administra l io * . 



* o, Resource*: 



State government in Olympia. 

EJucul I on, 1 1 television, taped lectures, loss travel. 

RcautLful, conducive environment. 

Specialists in area coul! he asked to teach classes m specialty. 

iO. Other convenes or sugger. L .tons : 

Members express appreciation of programs and the initiative that 
keeps them going, 

Kitsap County carrying out intensive course in coronary care. 
Course;-, given at University of Washington and Green River College 
have been valuable* 

Ve have no nodical education program. 

Training for paramedical personnel is needed. 



Le- . rn e_d_S_o_c 1 e l as 
7. educational Programs: 

Education for tied i cal technologists. 

More 2-uay nestings concent rating on one difficult area. 
Television program to show the value of therapy to multiple 
sclerosis patients. So far, amble to interest stations. 
Also, center for multiple sclerosis. 

Training, for emergencies. 

blood component th ropy, proper u L t lt;'.u L ion of blood and blood 
compoiv jtls . 

Lot lure*;, slides, and :ovi«s r )cv..u l to ophthalmology. 

V i •! cot ape . 

better vui Icty of c.Oj i- r.nindune topics r'i i .;,ilch ^ < and are 




CPMM!,::tS ( c < M'ljiviM) 



d, RcsO'ir' ■: «. s ; 

Pathologists roul'i be tapped f nr general physician cdnrnt ir>:i noro 

America;* College of .Surgeons has large film library. 

Tlie [Jr iver si ly . 

State, K?i! Cross blood brinks enthusiastic about serving,. 

Cadre of subspec ia 1 * st s prat: l lei ng mni ily but on clinical faculty* 

Largest lab In United Status for blood ccmpat Ibil ity , nearly largest. 
Kli lab, third 1 argest blood bank. Only blood bank with cuinp] etc 1 y 
cent rail red community* 

10, f'thtsr comments or suggestions: 

Would it be helpful for state medical societies to have lists of 
pathologists who are "exports" so as to spread teaching load? 

General meetings no longer do adequate job. 

2-3 day meetings are best for keeping abreast of vhnt is new. 

The disproportionate tlxc spent on general practice in view of the 
dre.reas lug number is not doing the job. 

Travelling teams educate at "grass roots" level. 



Con n t y Health Off ice r j> 



7 . Ld u c a t i ora 1 Progr an s : 

Any nev? knowledge from Seattle; labs need closed circuit tele- 
vision; specialised teaching personnel, 

Sex education, alcoholism, emphysema, rocogr.it ion of orthopedic 
problem , 

Our society could benefit fron any program. 

Venereal disease, tuberculosis, glue sniffing, hallucinogenic drugs, 
narcotics, also better uti J Izatiun of local allied services-- 
child guidance center, crippled children’s clinics, etc. 

Office preventive medicine, n ;w medical hazards (contamination, 
d ru g r , , c o sne tics), 

Part-time postgraduate training for Public health Hur-es, 

Family planning, lab methods in Cf>« 

b. Resources: 

Specialists in radical and paramedical state departments (Health, 
Welfare, 

Or t ho; ed f st in crippled children 1 ;; clinic. 

Pino television MoaJcast on Channel •/ (.-ducat oral channel, 

Yur.ima) nr.rc a vas-k at 7! 0:1 ,i.n« 

Facilities of Health Oep nr L'lunt (Jah, etc.); facilities of hi irk 
CuJ U-.;c. 



co:*rn.Nj_s (coin inued ) 



10, Ot ho r c orvn c i ; t r. an d f ; u p. tj o <; t i o n s : 



Keep it op.' 

Our society is alarmingly apathetic about programs. 

There is ci definite lack of knowledge among physicians coming to 
public health about the t , laws repu Lit i up, public health, ard the 
valuable resources through public health. Local educatxon scores 
to be helping ,;o;ne. 

Great difficulty finditp speakers on sex education, VO, drug abuse, 
etc., which are vitally important, 

With so few doctors in the county, a program is much too expensive 
to be local. 
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013JLCT1VJ2 3: Determine the types of educat iona J programs which should 

be developed to meet the needs of the physicians. 

During the course of this Contract, a significant change has taken 
place in both the philosophy and method of developing educational programs 
for the practicing physicians. In broad terms, the criteria for select- 
ing and developing programs are now being based more on the expressed needs 
and requests of local physicians rather than upon the desires of members 
of the medical center or academic community, In point of fact, many of 
the programs offered under the guise of continuing education have been 
up to now really exercises in "soft sell u — and occasionally ’'hard sell"-- 
persuasion of the attendee of bis ignorance or incompetence and of the 
speaker’s unique expertise. Pul differently, they have often represented 
the promotional platform of new specialties establishing their place in 
the sun or medical centers recruiting patients. 

Partly based upon the results of some of the course evaluations we 
hive done, such as "Urology for Non-Urologists," the content of some of 
the programs has shifted from concentrating upon evaluation of limited 
scope to that of content evaluation, which is seen by participating phy- 
sicians as review, reinforcement, anu the updating of knowledge. Many 
courses are making more use of small groups for discus; ion and individ- 
ualized instruction; there is less emphasis on the large continuing medical 
education short courses; and there is an increasing use of the problem- 
solving technique of instruction rather tl an a series of short lectures, 
one after another, during a two-day course. 
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An example of a course 



incorporating a variety of methods is the "hemorrhage and Thrombosis" 
course, where four different teaching methods were used-- lecture , 
problem solving, small group demonstrations, and panel discussions. In 
essence, areas of "wants" and "needs" are increasingly being converted 
to educational, programs packaged m a variety of forms permitting the 
local physician to select his own method of learning. 

Several of the recent programs have been planned by non-l/nivers i ty 
individuals and local commit '‘ees. For example, a cardiologist from the 
Central Washing ton area, Dr. Richard Twiss, examined hospital records 
in Yakima hospitals and determined that there was a need for upgrading 
the physicians 1 abilities to intovp et ECG's, particularly as they are 
related uc arrhythmias. There was also evidence of missed diagnosis 
and inadequate treatment of arrhythmias. Resulting from this deter- 
mined need, television programs were specifically designed, visiting 
cardiologists presented seminars, and hnndbookr were printed to meet 
the needs of this area. Subsequently, Dr. Twiss has taken it upon him- 
self to train physicians in outlying coranuniti es * Several physicians 
are now coming into Dr. Twiss* hospital via prcceptorship-typc programs 
to update themselves. This activity is a product of identification of 
specific need within a community by a member of that community. 

Commit tees comprised of other physicians from the Central hVisl ling- 
ton area have identified needs in the area of heart, cancer, and stroke, 
and have requested programs designed Lo meet these specific needs as they 
perceive t her.. A scries of three programs in each of these areas have 
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been vi d^oiape d , followed by a "vr up -up" session with the lecturers for 
discussion of specific areas not veil understood, or for extension of 
the content material. Those programs are now being made available to 
physic i a ns throughout the entire state. As a result, continuing nodi- 
cal education activity has been stimulated where none existed before 
because a small community tried to meet its own needs. 

At Green diver Community College, located a few niles south of the 
Seattle medical complex, a group of local physicians spurred on by 
Dr. William Shaw continues to attempt to iJontify their continuing medi- 
cal education needs. The contents of programs and top’cs are being 
decided upon b' f local pnysic i. lh ’roup, in turn, then requests 
personnel i rom the University of Washington and/or from private prac- 
tice to develop programs that would meet the particular expressed needs. 
The community college is being used as a resource in the Green River 
area. All of the courses and seminars are channeled through the com- 
munity college; they ai'e held in community college facilities and, to 
a large extent, are administered by the adult education division of the 
community college. In addition, the college reminds attendees of each 
of the sessions by telephone to Iris w i f e — it uses the sane technique 
regarding educational television programs — and conducts simple question- 
naire surveys of attendees af Ler each course. Those studios were insti- 
gated by the Project Director. of this Contract and feedback of comments 
to course organizers and individual speakers has proven part irul ar 1 y 
beneficial to those planning subsequent courses. Nmvover, review of 
the outcomes with the attendees then-eivis has helped to sharpen the 
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def ini L feus of their own objectives. Similar efforts arc: getting under 
way and will soon be initiated in Tacoma, Washington. 

A pilot precep torsi) ip program was initiated about two years ago. 

This program involves the participation ri a local physician in a one- 
week individual i zed program ,/ith a specialist in the area of the local 
physician's expressed needs in the Seattle medical complex. The physi- 
cian identifies the areas of weakness or the areas of study that he 
feels essential to improvi 2 ?g his practice; he then is matched with a 
specialist in a hospital setting thaL can est meet the needs that he 
has expressed, Evidence of the popularity of this program is that, 
presently, there arc more than 1,200 applications Iron physicians 
throughout the state to participate a precept orship program such as 
this. Plans are in progress new to use hospitals in Seattle, Spokane, 
Yakima, Wenatchee, and Tacoma in an e rr ort to provide learning environ- 
ments for those physicians who have indicated a desire to participate 
in such a precept or ship program. 

Participants in the traditional continuing medical education short 
courses sponsored by the University of Washington are consistently" asked 
to indicate what their needs are and what kinds of programs they feci 
would Ve essential to keep then updated. The responses are then ana- 
lyzed and an attempt is made to develop programs to moot the expressed 
needs. The following are examples of specific programs which can be 
cited that were developed directly as a result of Lhe participants’ 
expressed needs, examination of hospital records, and requests fvem local 
coiMTii 1 1 < o'* : 
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The Bedside Diagnosis of Stroke, 

2. Ancillary Method., in Diagnosis of Stroke, 

3. Recognition and Treatment of Arrhythmias, 

4. l.’hat's Men in Endocrinology? 

5. A Review of Gas troenfcerojLogy--vIo.dic.il and Surgery. 

6. Is It Neurological , Neurosurgical, or Psychiatric? 

7. Preceptorship programs related to arrhythmias. 

S. Animal laboratory experiences arc now available three times 
a year, for 15 physicians to gain skill inserting pacemakers 
on a one-to-one basis with a cardiologist. 

In summary, a greater effort is being put forth to organize con- 
tinuing medical, education programs around the expressed and/or deter- 
mined needs of the practicing physician. Several such courses have 
been cited. The importance of incorporating a variety of educational 
methods instead of n series i>f consecutive lectures seers to be catch 
ing on. As needs of the practicing physicians continue to be : ore 
specifically defined, objectives for continuing nodical education pro 
grams can be more adequately developed resulting in programs that can 
be designed to achieve the objectives. 
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OUJhC'nVh 4 : To develop a system f «.> r evaluation the of f tc t Hv ness 

of those programs. 

A great deal has been written and much more said about the need 
for evaluation of continuing medical education programs, yet the 
evidence of effect iven/ss of continuing medical ethical ion on medical 
practice continues to be elusive. 

As we have worked on this phase of the contract for the past 
several months, researched the literature, and talked with other people 
cop.ccrnud about this aspect of continuing medical edr wit ion, tee have 
come to accept the following conceptual framework for evaluation. 

Six premises upon which to build a system of evaluation are: 

1. The purpose of evaluation is to judge decision 
alternatives . 

2. To apply criteria to decision alternatives, it is 
necessary to hive relevant information. 

3. Different studies require different evaluation 
strategics. 

4. Different decision questions require different 
evaluation designs. 

3. While the substance of different evaluation designs 
varies, a single set of genera 1 liable steps can be 
followed in the. design of any sound evaluation. 

6. Since evaluation studies should answer decision 
questions, evaluation designs should satisfy the 
criteria of practical utility as well as criteria 
of scientific adequacy. 

based upon those premises, we night define evaluation as n the 
procc:>3 of def i n l:v; i obtaining, , a..J u sin, infer, at Jon to Judge alterna- 
tives* 1 (5itu f f h h.s'o , Ibbb). Sih. definition i:id li a t . r* that cvr. hm H on 
is a process. The process is a * on* hmir.i; *uu: and ray l. 1 represented 
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ass«i!”.i lv; that the dofinlii 
objective:, anJ criteria go on continually tn a sequential 
constant review and revision of each slap. Tiil-s approach 
definin'; i nforrt U fen to b fi coll err e<j , oh t a i n in;; thin infra 
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utilising the obtained information. The information gathered is used 
to assist in nuking decisions among ava i lab } c a 1 1 orna t Ivc s . The scien- 
tific criteria must include internal validity, external validity, and 
reliability. In addition to the scientific, criteria, the evaluative* 
Information should meet utility criteria, such as relevance, signifi- 
cance, scope, credibility, timeline?;#, pervasiveness, and efficiency. 
Kith this conceptualization of evaluation, we have proceeded in 
the fulfillment of tills contract to approach the evaluation of continu- 
ing radical, education programs in such a v.y that the criteria for good 
evaluative information are met. Since \.v arc working in a very real 
world, obviously we were not always able to satisfy these criteria. 

The approach that we have token on the var’ous projects is to work 
with tiie project director of the sponsor of tnr cmrse in a team effort. 
It is assumed this approach will yield t.oiv s! •uificant results than 
the approach of having an "outside ovalua t >r' r iV.di.Mto a program for 
the Individuals responsible for it. 

Lx ample:* of the approaches we have used in building evaluation 
design for continuing medical education pro. -jams are included herein. 
Those exn pt»m inclule the Central Vasil i n \\ er. Vroject, the Coronary 
C u*r b’nit Lroj/rt, "jk moirb.gge and Thro-Vj-ii".," uni a pilot tlo-unl- 
intif-ji f.t-idy, TI. " approach used involve. : iv. i,‘d steps. Klimt, the 
evaluators not with those responsible for lie' course or t ho project. 
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either instruments readily available or instru .cat s that wou 1 .1 be con- 
structed for this particular project. Secondly, alternative research 
designs wore presented. l'ach design wos discussed in terns of its sci- 
entific merit, s.V’pJ inp procedure, data-co] Jeetina procedures an l gen- 
eral acini ni stra.t ion procedures. The* Jnterpr- tat ion of results one 
would pel from cacti desfpn was discussed in t.cnr.s of the decree to which 
generalizations could be made from this study to a broader population. 
Acct ppanyj np the discussion on research deslpn \n;; our recorders-! a t i on 
re p a r d i a p w; \ l ch u ae to foil uv* , 

Those responsible for the prop. ms would react to the various 
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Kxar.nl o 1 

proposr.n firm:;;; top ckt] • at. i:ASiir:;rio:; stkokk pkoc;ka“ 



01 ->Jo ctr. i v o 

1, To proviso opnor ttini. t i cr, for 

t c; nch i n n and 1 n a ru 1 n p vh i ch v i 1 1 
improve care of victims of heart, 
cancer, s t robe , and related di 
eases. 

2. To improve corr uni cat i on between 
the Seattle academic cor p lex and 
Y ak i 5 1 1 . a w\ licit w i 1 1 a* u p,rr nit c on - 
tinuinp education. 



3, To inpro\ communication be- 
tween Yak inn and its surround- 
ing commit i cs in ways which 
will augment continuing edu- 
cation . 

4, Stipulate health practitioners 
to identify their own edu- 
cational needs. 



3. To invest 1 pale records or what- 
ever pertinent data are avail- 
able to subs t an 1 1 a to nerds and/or 
determine others, 

6, To desijn personnel and facili- 
ties to pain rax I: u: profit 

f r o: lies p i 1 a 1 visit s b y S e a t - 
tic academic personnel. 



Kva 1 nn t ion Snipes t i o nr. 

A on e , Ob i o c t i ve r r e r - u r c d rot 
with presen tati on of nrn^rnw. 



Uv having lira, Fryer and Titus 
Keep a record of information nl 
c omrun i c a t i ons f r or: Y ak i na a re a 
starti up noi/ and continuing 
through Pec. 31 1 1968. The re- 
cords will be kept on fortes pro- 
vided by Ur. Dohncr (see attached)* 
A follow-up questionnaire 
will be devised which will in- 
clude which proprana were 
watched as a question and also 
yield information as to how 
many physicians watched. 

Dr. Gustafson and any others in 
Yakima he feels appropriate 
would keep records start in r. as 
soon as f orms are sent to them 
and continuing through Dec. 31, 
1968, In the manner above. 

Perhaps this can be ret in part 
with a question on the post- 
proprnn follow-up questionnaire 
used for objective 3. (lo be 
sent out and collected by hr, 

Cus ta fa on, ) 

Tone , Presumed done by II :K 
and prior questionnaire sent 
out before the prop van bo pan. 

Kcao . y.ct with* <’ ra; -up sessions. 
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7, To design evaluation instru- 
ments which will reveal prac ti- 
ll lone rs 1 p rog res s l own r d Ron 1 s . 



A, A pre- ant! post- Lost cover- 
ing tele lecture, content aci- 
ni n i s t e red to a s a pip 1 e of 
physicians in the? Yahira area 
and surrounding cor nun it ios. 
Vais requires: 

1 . A list of p o ten t i a 1 
viewers , 

2 • As ar p 1 e of physician s 
f r on a si ni 1 ar c o nr unit' 
for ins t runout relia- 
bility. 

P. Administering a Semantic 
Differentia] attitude re ns u re 
(see attached) along with 
the pro- and post- test to 
me ensure attitude toward the 
telclcctures and the program 
generally , 

C. An examination of hospi tal 
records covering a period be- 
fore and after the program 
using criteria established 
by Drs, Fryer and Titus in 
order to see if any behavioral 
change in practice, tabes 
place following the program. 
Possibly using nod, students 
to rather this data and also 
to conduct short interviews 
with Drs. who watched the pro- 
gram for opinions. (A pilot 
study would begin this stirrer 
as soon as personnel and re- 
cord availability are ob- 
tained.) The data would be 
examined the foil nv i n g s ui : r e r 
depending on the results of 
the pilot study. 



8. To re-create habits of self- 

instruction. ? 

9. To provide latest advances in Done. Presumed ret with Iccl urc: 



the diagnosis, 


treat rent , 


and 


rehab i 1 1 1 at ion 


to vie lira 


of 


heart, cancer, 


and strobe 


in 



Central Washington area. 
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methods or' evamv.tiu:: 

1, data relating to: 

a. patient care 
h , p Va c t i. t i on or n a r t i c i v a t i on 
c, chanpoi; in practice patterns 

2 , Pe r ~ oil i c ex an in at! on r? of: 

a • hospital re cord ;> 

b, disease nor tn 1i ty-norh i d 1 Ly 
data 

3, Demand for self- instructional 
equipment 

4, Depren of viewer response, 

5, Participation in summary 
discussions, 

LIST OK DETAILS 

1, Names of physicians who will likely watch the program, 

2, Use of the Semantic Differential, 

3, Sample for pro- testing of knowledge pain instrument, 

/*, Tunds to support r.edlcal students (wapes and per dJcn) for record 
study, 

5, Approval of instruments and procedures. 



record study 

foil ow-up r j uer, t i onn a i re 

record study 



record study 
? 
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follow-up questionnaire 
head count? 
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A'n i ti t i »t . cmaya:: y.r/.suhh 

Con si dura t ions : 



The Scran tic Differential is a device for measuring attitudes to- 
ward a construct and also the relationships of the construct, con- 
cepts to t"ch other. 'Hie most commonly used scales define the 
construct selected on the dirctmai ons : evaluative (pood-bad), 

potency (weak-strong) , and activity (active-passive)* 

Thus a concept such as 11 television 11 could be seen as strong, 
active, and potent or as various degrees of these factors. It 
could also be seen in relationship (i.e*, where it stands with 
all other concepts being measured). The overall evaluation of 
the concepts defines an attitude toward the construct (here, the 
Yah inn Project). 

The purpose of the Differential is for attitude no as vireTr.cn t and 
it nay bo used to test attitude hypotheses. It is suggested that 
one way of measuring the of fee t i veness of a continuing medical 
education program would be to measure attitudes tcv^prri relevant 
concepts of that program befor • and after cho program 

The following are examples of what are felt to he some relevant 
concepts to continuing medical education attitude, specifically 
for the Yakima Project : televised learning, television, lecture, 

televised lectvirc, review material, research material, con- 
tinuing medical education, practical material. 

The research hypothesis is that there will be a significant change 
of physician attitude during the course of study: .specifically, 

that "televised lecture." "televised learning*" and "continuing 
medical education" wi 11 come to be seen as more active, stronger, 
and better. 

hr fool that it is aj.se important to sec whether participants’ 
attitudes toward "television," "lecture," "review material," 
"research material, " and "practical material" change as a 
result of these programs. Other studies, such as "Urology for 
Non-Trc 1 ogJ *» t s , " have shown d iuagrcor.vvit about these concents 
and continuing lied leal education. Tire data would nrovide guide- 
lines for the planning of future programs by serving to 1) identify 
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below i:; a ti’ru v.iich can be ral*. <\ by piac In;; a:t a in the pp.'kv which 1 ■■: 
represents yo^r Ib.-eling ab-nif lit it. tcr;i fr*r various pairs of vors. Is* 

l itfil. so 



vo.r y 



very 



good : : : : bad 

active : _ : _ L : • passive 

veal; _ : : ; ; : : _ __ strong 

First, you vault! decide whether you feel HOUSE so ms "good” or "b;uK ir If 
you feel HOUSE, way very "bad," you would place an X in that spnee, You May 
place an X wherever you feel *it should be to represent your feelings abo.il the 
tern HOUSE . 

While tills may sce;n difficult, it has Hern shown th.it such ratings of tor 
are possible and reliable if done rapidly, without drolling on each scale. It 
is then possible to get a picture of bow each Lo nr- is seen with regard to tb- 1 
various pairs of words. 

On the following pages are several terns. Flense work rapidly .Dai Murk 
each Lt/n n every s"ole. do not o:nii any teres er ratings. 
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To 1 ov 1 ' i * 'it is: 



very 



light 
bad 
largo 
clean 
fast 
veal 
ugly 
passive 
s In r p 



heavy 
good 
sira 1 1 
dirty 
slov: 
r.t rong 
l)C*3Ut I f u 1 
act ive 
dull 







Eer tun r* ere: 

v,-ry very 



fast _ _ : 

u;\\y 


l . - - - - - - - ■ 


: slo-v 

: br/viv 


large • 


, 


: sr.a 1 ! 



active : : * L : : passive 



strong • 


: : : : : wea k 


dull : : 


: : : : sharp 


dirty : 


: : : : : clean 


heavy : 


: : : : : light 


good : 


: : : : : bad 



o 

ERLC 

hfflinaffamiaaa 



" V 



Tt* 1 o.v 1 r.»'J 1cc!'nrt , '» arv : 



v 



r y 



f Mil 



heavy 
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dull 
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clem 



slo-.: 

active 

li^ht 

larj;e 

sharp 

bad 

beau Li fu 1 

weak 

dirty 
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the eta si ret. of t - ho noTioncr, 2) specify the cor.nosi t Ion of the 
audience with regard to their attitude toward the purposes and 
the present at ion of the program. 






Hie homan t i c Hi 1 fe rent i al is thus a flexible and demons t rah 1 y re- 
liable. device for lrcr-surinr attitudes. It is felt that attitudes 
are one indication of the effectiveness of n nropran such ns 
the Yakima Project. 1 he scale is completed rapidly (anprox irate ly 
15 minutes) and the data can be dealt with an on y nous ly . 

1 1 , KN T 0\ T IJ-:i jrE GAIN 1 rMAS bhi; 

Con s i de ra 1 i on s : 



1. There must he a base rate group which docs not attend which 
is tested before and after the sessions. If necessary, the 
comparison could be made on two random groups fro™ the popu- 
lation of non-participating physicians. 

2. There must be a pre- and post- test demonstrably equivalent; 
or the same test may be used if test- re test reliability is 
shown to be high* 

3. If the same group can be tested before and after a particular 
session, or series if possible, then the data can be evalu- 
ated in terns of net gain* 

L ' . Ideally, the groups would be stratified samples; however, 
this seems to present too many problems of data gathering. 

5. All designs are for one session or one series; they would 
have to be repeated for each area, o.g,, erncer, stroke, 
heart, if information is desired for each, 

6. Preliminary test construction should be done on two groups 
separate from those who attend and those who do not attend 
s amp lea, 

7. Vhether the physician watched at here or at a hospital, what 
type of practice he en paces in, ape, date of graduation, and 
postgraduate study should he considered In evaluation. Tims, 
a third design would he analysis of covariance or randomized 
blocks . 

8. 'Jko period after the sessions before nest -t es t Inp determines 
whether ve are measuring immediate recall or permanent 




D J 



30 



knowledge gain, T hus , a decision rust bo made for when post- 
testing mil 1 tfdco place, 

dksic;; i: 

A random sample of the non-parti ri pat ing physicians for nnv pi von 
session, or series tf possible, is to be adnini stored test a,, A 
random sample of the participating physicians for the same session 
or topic v:i]l also receive test ji prior to the session* After each 
session, a random sample of participating physicians, as well as 
a sample of non-par ticipatinp physicians, will take test b_. (h'oro: 
The same design would apply for n test retest using the sere instru- 
men t , ) 

DKSIGII 2: 



A random sample of non-partici pat ing v physicians for a given session 
(or series) will receive test A random sample of participating 

physicians would also receive test ^a. Both groups would be tested 
at the end of the session or topic* 



DKSIC: ’I 3: 



Data on age, postgraduate work, viewing conditions, practice, and 
date of graduation would be gathered for all physicians partici- 
pating in the stud.** Pre- and post-tests ’would be administered 
to those physicians attending a given session {or series). Also, 
a control group would be measured and denographic data gathered, 

A randomized block design would be used to evaluate the results. 

Design 3 is the nest preferable, design 2 is next preferred, and 
design 1 is least pro ft ’red from a statistically defensible view- 
poin t . 

Ill* CHANCE IA PRACTICE KEASVRK 
Considera tions : 

1, Cooperation for access to records would bo necessary from, the 
hospitals serving the central area, 

2, Criteria for desired diagnostic tests and treatment:! would 
have ,o be established. 

3* It would be necessary to know which doctor; attended each 
lecture. 
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4. A staff won lil bo necessary to search the records, 

5. Ho i' n ovled re testing would be necessary to evaluate the 
effectiveness of the project. 

n ihlXX : 

Hos p i t a 1 records 'would be examined for a period prior to the 
series for patients whose final di times is for the ndr.it ting symp- 
toms is known. Histories, tests, ami treat rents ordeiod would be 
recorded. Yhe results could be assigned scores based on criteria 
established by the lecturers. 

In order to derive criteria for assirnine scores, it would he* 
necessary to r.cot with the lecture r(s) Cor each, program, 1 ani- 
mal ly acceptable practice and desired practice levels as well as 
the. relative importance (wei elan) of the various parrs (i,o., 
history, treatment, and tests, etc.) would have to '.>■ ascertained, 

Thp first examination, of the hospital records could be sin as scon 
as the fine! propram had been written and the criteria estab- 
lished . 

At a determined period afLer the series, the records would again 
be. examined usinc the same criierin. The scores before the scrims 
and after the aeries would then be cor pared* Higher scores 
following the series would indicate practice ch pages, 

The relationship between number of lectures attended and practice 
change could be examined as v’oll as ether relationships do terri netl 
in advance provided the data are accessible, 

iv, coiyrxiCATiox i;vam;ai rox 



Objectives 2 and 3 for the Yakima Project are to improve communi- 
cation between Seattle and Yakima and Ynkira and its sv.rrou.id in. a 
communities* 



Vc fee 1 that four typos of 

1. telephone, 

2. letters. 

3. personal visits, 

4 . ref u r r a 1 1 e 1 1 c r s . 



neaaura 1 ' .1 c cc ma un i cat i on are : 



Increases in f re cinerary of these eon mice? 1 1 on channels could he 
interpreted as being related tc the Yahir.i Prelect if a base 
rate count ware established lei ore the series for rer pa rat i w 
purpose s , 




i) ? 



No thod 



r e 1 o c l a s l 1 ,v 'T'. 1 e 
wlio are reform! 
ouch soriiv- and 



of physicians 
tnrpe.tu, A 
voul d i nc 1 ude. 



in the. Seattle anti V.^irvi urea: 
sample would be* selected fur 
the series lecturer (a) • 



2 . 



If co one rat 5 on con he obtained 
(count) the nnrher contacts 
methods of cor muni eat ion . ihi 
and continue for three months 



the secretaries would record 
for each of the four above 
s count would start immodi ate 1 
foil c-; \ a e the particular c r 5 o 



3 , 



Record hue for ms and instructions '’ill be 
secretaries hv the Office of Research in 



suopl ted to the 
t 'odi.cn 1 Vdncatfnn, 
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cat;; i tcc \vj<m v. ihiv.nxv 

lii | i ; ;o / p.u'r t » T thf ; project Is io mi; art and ecv ! i . : < 

Si'l< -TV.- , A'-; ! ( it -y< : /. . r )■ i'. :i «. 'An r.f hde« t i on ; re-,1'. e to tr* p.-.re e e; ■ *’» ■ . i 

r :• i f * : 1 y A / 3 cur ;-,es j».*r Mr the* next tvo yc. ;r;j i 1 1 t ; 1 1 l. r .* : h : 1 1 ■■ ■ i ni,~ 

A ins ha tsyAei t «' f r rioi MCi: in Coroii.ji-y Car.- Units. 

A .'it.', nil'.ird j r»od rii'rif.'uJir; will provide a K j .O-hour c:o!.'r;(', Ind :d i 

VO hours of pi. j; t in j mi and 'TO li j r s o r nurse I n-: L r no t rj r time, It rill 

include a mixture of didactic anl clinical 1 *: amir.;; e/pe i* i i :e k es . V’;-' 

project v. r £ 1 ] Le uc.ec: in! i sh . d with i nv I \ r c^ ^ * r. t of ceuvaunl L y colli yes, 
state uni wr si t ies , ToiyjI physician and nursa groups, in- .or c ice 
directors of local hoepitvi ' s, an.! the UYadi trig ton a. id Alaska htrte hear 
Assoc intioi’fi . 

Through the h’a shington/Al askvi Regional Medical Program Corn- a y 
Care Unit Coordination Project, feedback from the suh-rug ions has made 
It increasingly clear that rduoat tonal programs of excel fence mist 
move to the suh-reylons --the sub-rej; tons cannot go t o centra lira J 
training centers’. Tin.; project is designed to provide a beginning 
frames nr It for development me! improve rmnt of cooper, it ive networks with 
in and between suh-ropi onal con^on i t. i os , their health care profession- 
als, and institutions in the development of ongoing self-supporting, 
c o n t . f m i i i ig nu r s i n& o d u c a t i u n • 

The evaluation which has been prepared for this project, by 

our office Is as f o flows : 



CORONARY CARR UNIT hVAl.VAT KKi SaiKMK 

I. ProM im : 

There exists, at. present, in the state of hashing ton a need to 
train Coronary Cam Unit (CCU) nurses on a large scale of opera- 
tion which is bo tn locally operated, yet centrally nonitored, 
and partially coni rolled for quality. 

In older to maintain a dcsiied level of quality, it is necessary 
to evaluate the process and predart of each program us in; cri- 
teria rst a Mi shod centrally by l he CCU Project staff in conjunc- 
tion with til- Office of Research in Medical education. 

I'.valu !t ion, In turn, r«ur:t mecL statistical criteria, such :e; 
defensible sampling pi oc edui’u k , reliability and va 1 ! d j 1 y — e t min 1 
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procedure , 
ll * L V J 2 X'}'. [ : A ). : 

It is sii<;s;i , :.lu! t Ii-n f the c\\»1 m.u*? on of the C('U Project will include 
measures. of kjuu/led-e, nkill, attitude and confidence in km.elmh^ , 
a nd tint nee e : : sn r y 1 eve 1. u r f at tat iv: cut in each of 1 1 .e c* four a r i as 
be det crmlned In advance of Lie.' onset of ;lie programs. 

A sens'd. i on'; : 

A. It is assumed tb.it there are uiuii.uiT.i level*; of V.novl 
and uki 1 1 necessary to 1) be of essential use In a (XV 
and 2) avoid preventable barn to patients*. Those Lvr> 
clauses are cor.no neats of post-pro;*. can effectiveness. 

Only essential knowledge and skill will bo included in 
determining the-se minimum levels, 

11, It Is assumed that tendencies to hold certain poslLive 
attitudes favoi’aMc to CCU work are move desirable than 
n or, a t i vo t e nd e r. c I c .■> . 

C . If is ij.su sed Ihm confidence is an atrilate of post- 
pro;* raa effectiveness (as defined in Assumption A). 

III. Hy pot banes: 

A. It is liy pot lies i ::ed that attitude and knowledge end skill eill 
correlate positively, l.e., teal those po'' sons’ with lsu'c pos 1- 
t ive attltulc: v:i 31 do IhiIUT on es M utlal and iien-ossn nt J 1 
kmr;le<3,:e and skill measures than will equally ot be j mi :e o - 
potent person:; as r. insured by pie- pro .-rai \ knowledge 1 1 v « ■ 1 *- * 
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s. It i ; hspoJb d y.\ tVW r.'i'v.-:; who r .1 1 ^ 1 as :Tpi r:r^ T .\. 
of fen t lven -r; by jm - y :t 3 • i r ; ‘ J ? r« or nf :h>: unit si 1 : numbs 

post. - proy.JV *" ri 1 1 h'.-V 1 ' v-uv v _ .'.r 1 : r« '«! civ:?* Mr iii'-’ »h';!> the.;,-, 

names '.‘ 1 :*' r'! i * * ol I: 1 r ■: « ; o e'p'nHv e« .mj h’it who m <• relis 

r. deco at* < i i * d . t i • 1 1" , 

IV. r, \y i t . - • : 

A. beeno M> t k I s pto.\r.?‘ * o 1 1 ; : i : : ^ * 1 1 1 tl * L* groups r; i ai* *: I :n*. r : 1 5 f” f t - ? ■ 1 1-“ 

ip;’, t 1.0 it pV< , ; r ,Va , ..V; * j l V t Li'i'S , OV :•.’»! 1 1‘ 1 on wi M not h" 

p/'iior;.j l i :* 1 1 ) 1 *. ta other roups „ 

15. The ova] u.it Ion is not p.oiny, lo tot for 1 * l rc.it j.'ir-nt , 11 let rather 
to sv.n if the t ; icc I r led objectives vero reaHved, l,e fc , iln the 
CCl! program products K.ve il ssin d knowledge , .skill, and nttUudo 

C, The evaluation is not {-nil'.;; to tent lust rue tors per so, bat 
rathor to soo if any programs are r.oro effective (have hi nov 
success- rates) than otl :ers and Lo uiflp.cst hypo t noser, ns Lo 
why this is, 

1). This evaluation thvs not include* tho tapnet on patient care, 

V. Inst rr.nent s : 

A. Knowledge vill bo measured usinp, r.ul t ipj c-cboico anj recall 
itons. hi eh knowle lye item will bo acc.onpnnicd by an indica- 
tor of confidence in response from 1 (confident) to 3 (I don't 
know), Doth essential knowledge and non-essential knowledge 
will be identified and measured, 

Y>f ekill will be no. i sored usinr* «' simulated problem in a CCLT noci;- 
vip . Specific operations will be "checked off" as done or net 
cl one- -non- en; ?nl 5a ) and essential skills will be id* nM'.Mi i and 
measured. A pat ient rinna^e;; ent niwane riiplit also he coa i do rod 

C. At l I Unlc '.’ill he neasurt'd iisln*’ ell.her a sonant Lc differential 
(see attache! r.\\!i:;pie) r>r Likert (five-point) scales ov some 
other ruthed lo he deli mined before the start of the pre.-r, ■ 

1), "Ceiif /draco 1 will he rs usured by a five-point r«c- ale ranyii,'. 

from ’V<uif l.h at " to "I don't h»ov, M and free + 4 to -h f 

i.c., a pen-en who say:; ho is "confident 11 an I ;;ror» i> ; i nerr ivt : J y 
receiver! a -4. Such a procedure '/ill not confound th- 1 ]■ , dye 
no. I sin*i*s hoCAwrv the au'.V'i-r:; re ’.tin the; suhjcet.s 1 best p.ie-.-s 
in any event. 




bi 



Vt. Proven’-! re 



B. 



C. 



]) < 



E. 



The Knowledge , confidence, and skill tor.tr; will no developed 
and pro* tested on r. group of nurses not involved in the project 
otherwise. The final instruments (pro- and post-) will be 
mailed out to c a c h program director with instructions for 
administration and return, or will he administered by a field 
coordinator or another designated individual* 

The altitudinal measure will be developed and mailed out alone, 
with the other instruments. It will be given pro- and post- 
prop, ran , 

The skill measure (s) will be decided on and mailed out at the 
end of the program in order to avoid contamination, i.e,, 
teaching of the instructor only for the final skill measure , 
which is only a sample (hopefully, a good sample) of behavior. 

Administration of the skill test may be done by or;e of the 
field coordinators or another designated person in order to 
assure reliability* 

A random selection of data from all of the programs will be 
made in order to test the hypotheses. 



Figure 1 below shows the 2x2 factorial design for hypothesis A. 



VII, Design; 
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kvai^atio:; Ri:i*o;vrs nun t:i:: con::::;, 

"KRMOIrKRACi, A.V) T IS" 

A different evaluation design was initiated for evaluating the 
course, "lie; jovrhugo and Thronborh . 11 We are quilt* opt ir.vlr, i i c about 
t be feedback inform." t ion that this method r.ade available. I be approach 
11 ado use of an evaluation team of observers f rota various academic areas 
which was comprised of two participating physicians, two technicians, 
two participating faculty members, and two education specialists, not 
associated with the medical school. 

Each team member was given a rather unstructured outline consisting 
of a few questions around which he could record his impression of each 
session during the two days. After the program, each evaluator sub- 
mitted a written report of his observations, Including re contend. at Ions 
for improving the course. A week after the course, a meeting was held 
with the chairman of the program at which tine eacli team member was 
given opportunity to share observations about what was done well and 
what needed improvement « The chairman also queried each, participant 
by nail in an attempt to ascertain his reaction to the program. 

The following reports are examples of the observations made by four 
of the eight team numbers: 
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KVAU'ATIO-’J Kj;POPvl OM THE COURSE 



' IlhMORRhAUh ANfI> THROMBOSIS 1 ' 
by John Priscoll, Th,P. 

Part I: Effectiveness of course 1 

A lay viewpoint on effectiveness is naturally quit^ limited. If 
one attempts to judge M cf fectiveness* 1 of planning, organization and 
presentation, then I would say the course rates high. Presentations 
demonstrated careful relationship to proceeding topics. There were 
ineffective spots; for example, the first presentation — obviously a 
clearly thought-out review of coagulation — was too long and could cer- 
tainly have been streamlined for such a short course. The panel on 
"Treatment" (11:20 second day) was useful and looked forward to by the 
audience, hut was reduced in effectiveness by an over-due in te rnissi on 
and by following another panel. At other points in the course, two or 
three speeches were lumped together and seemed a little tedious for the 
audience . 

Part II: Appropriateness of teaching methods for adult education 

There are characteristics special to medical education which arc 
difficult for the non-medical person to assess, bvit the following gener- 
alities about the course methods are offered: 

A. Projected teaching aids: 

1, The slides tended toward dullness of presentation due to 
lack of contrast in the first set and repetition in the 
two sets following. 
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2, The slide showings were almost ruined by the pcciiun light, 

A directional light for the podium which dees not spill 
onto the screen would be very inexpensive in comparison 

to its usefulness in improving presentations, 

3. The slides using colored letters on blue bach grounds were 
the most effective, but even these badlv needed some 
variety, I believe there is a tendency to use the many 
advantap.es of the projected 1 mage largely for words to 
reinforce words uttered by the speaker — this is under- 
estimating two great advantages of slide showings, i.e,, 
use for reinforcemon t via symbolic and pictorial repre- 
sentation of abstractions, and for stimulation of interest. 
These advantages should not be by-passed for the sake of 
traditional limitation to listings and photomic rographs . 

B» Speaker presentations: 

1, There vas a certain amount of inconsistency in these due 
to some speakers presenting too much material in a short 
tine and thus hurrying over some points while a few pre- 
sentations v/ere overly long, line graphs in the talk on 
prophylactic uses of ant i-coapulants , for example, tended 
to be tedious to look at and to lengthen the talk, 

2, The presentations would seem to be more meaningful if 
more use of statistical evidence were incorporated. 



III, Kec ommen d a t i on s for improving such a continuing education cours e. : 

A. General plan 

1, Streamline and edit material to fit into a fast-novlnp, 
cogent, one-day learning experience , 

2, Tlan learning facility so as to avoid large auditoriums 
for small groups. 

B, Teaching techniques: 

1, Improve slide presentations by employing more photographs 
and symbols. Avoid deep research- type graph presen tat \ ons 
such as that on megakaryocytes. 

2. Whenever talking about changes in blood biochemistry which 
arc visible through roll i, use the motion pictures avail- 
ah 1 c - - e ven a brief film-clip is < ttcr fYm a leap set of 
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slides if and when motion is inherent in the chnnecs 
being taught. Po not, either, un«le res titnate the intoru-*- 
stimulotion advantages of film clips. 

3. Use more clinical presentations, such as the panel on 
clotting and the case problems discussions of Friday 
afternoon. (In fact, using. the case problem technique 
earlier could make the learning experience far more 
involving. ) 

4 „ Pay more attention to pacing — less speech after speech, 
imre visualization, more bring-bnek-in-bri cf to the large 
groups significant points from demonstrations and small 
group discussions, less overlap quicker, edited reviews 
and talks. 

3. Consider variety possible by bringing in a pharma r Jtical 
person, as you did with bringing surgeons, 

6. Recapitulate briefly and make some quick observations 
of tomorrow ! s trends and research. 
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"HEWORPJiAr.!- AND r J , lmO^ , bCSIS ,, 



by William D. Koarick, IN A. 

I, Ideally, what should a short course such as this atter.pt to *.o? 

Confront the practitioner with data, procedure;, equipment be ri'h; 
not now be aware of, and provide bin sufficient opportunity to gain 
sore competence in ?lts use. This mean? L tat a highly pragmatic 
orientation is necessary. 

II, To what degree did this course approach the ideal mentioned in 
number 1? 

The presentation of problem cases, the case study snail £>i'oup dis- 
cussions, and the panels on diagnosis and treatment seemed to rent 
this standard in that order, although it might be a toss-up between 
the first- and second- named items* Both brought cut the most dy- 
namic participation of these in the course. 

However, in conversation with two of the participants after the 
last session (a technique for evaluation I would recommend for future 
use), both doctors to whom I spoke praised the entire two days. They 
labeled the case studies at the top, the problem cases next* Both 
did recognize the necessity for the other data-presenting lectures with 
appreciation. They expressed praise for the authorities who pre- 
sented the lectures and noted chat hearing then in person was much 
more effective than reeding articles by them (or others). They stressed 
the dynamism of the lecturers particularly. 

One r.c has attended several conferences this year but marveled 
that, in his area, there are so few doctcas who attend their local 
efforts to "keep up M or core to such conferences. He hypothesized that 
perhaps such a necessity wasn't nurtured in their Initial prop, ration. 

The other noted that he felt the brochure advertising the session 
misled hin — he suspected it night be Just another bunch of boring 
lectures, and couldn’t understand why some of his collciigues didn’t 
cone along too, except for what appeared to be a boring format. He 
urged the enphasis on the "practicun" sessions, e*g*, the ’'presentation 
of problem cases" and the "snail group discussion" case studies in 
advertising these to help others see the applicable value. 
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Session I: 9:00 a,m, t Thursday -- Dr, Davie 

A. Excellent organ! za lien , ideas moved one to another smoothly , 
logical ly , 



B, Evaluator not qualified to comment. 

C. 1. Effective: Slides on time; light control smooth; arrow on 

slides generally well used; conversational speech 
style. 

2, less effective: At times the arrow on slides was too active 

and distracted and at. other times it covered the 
material speaker was talking about; two sets of 
numbers (Roman and Arabic) confused both speaker 
and audience at times; slides were not consistently 
li^ht-dark or of same type face which forced readers 
to extra strain. 

3, This seemed to be a statement of current position — vhat we 
now know. As siHi, it seemed logically essential as a point 
of beginning. Although die speaker had to shoot in the dark 

to guess what his audience already knew about this, from general 
audience receptivity (and conversations with participants) it 
seemed to hit the mark remarkably well. 



9 : A5 a.m. , Thursday — Dv, Rapaport 

A, Organization seemed adequately smooth anu orderly, 

B, Evaluator not qualified to comren*. 

C, 1 Effective: Started by exhibiting a sense of humor; ho handled 

a forgotten slide easily without distraction. 

2. Tx'ss effective: He seemed to be racing at the outset although 

I got used to it gradually. At the start I felt 
that he wondered if Ills "review" was necessary be- 
cause lie alno, t saemeo bored with giving it. His 
use of the arrow on the slides was very global and 
distracting. He looked often at Dr, J/avic and Dr. 
Hougie as though he felt he were chiefly concerned 
with addressing them* 



10:30 a, m, f Thursday -- Dr, barker 

A. Didn't seen to be a matter o r organization; ho merely presented a 
rationale for the manual. 
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Session II: 2:00 p.m. f 'Huirsday -- Dr. Darker 

A. Even I was able to start following vi th some understanding, 

h. Evaluator not qualified to comment. 

»\ 1. Etfe:tive: Slides excel 1 lit and well used with arrow generally; 

excellent sum iry; took less time than allotted; 

2. Less affective: Drew in items from morning sessions to good 

use , 

3. Ecss effective: There seems to be an assumption on the part 

of all speakers that nil these participants see 
what they see in the slides and in the charts; 1 
wonder if the participants road such data as con- 
stantly as the speakers do and therefore don't 
need help in locating specific material as quickly 
passed over by the speaker. When obviously nev; 
interpretations were at stake, Darker did take 
time to explain. 

Right after lunch it's pretty deadly to have so 
much dark auditorium with so little active audience 
participation required. 

2:30, Thursday -- Dr. Gardner 

1, Effective: Somewhat conversational in presentation; nicely 

outlined introduction to entire presentation on 
first slide helped the audience see where he was 
going. 

2. loss effective: Some slides had too much on them; the material 

not used in the presentation seemed distracting at 
times , 

3. This presentation was lens research oriented and more theo- 
retical. 1 wondered how the participants reacted to Mils 
shift (although they were obviously hearing from a special- 
ist’s theory). Do they want un- M usable M theories or data they 
can trust? Or is it an objective to push the thinking of 
the participants to consider theories of their oeai ? 

In the discussion, questions concerning treatment persisted. 
This raises the question of quantity of practical data — how 
much should they get or should that be a matter for thorn, to 
apply later? If so, should they bo aware of that expectation 
be f ore they come? 
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B, Evaluator not qualified to comment. 

C, 1. Effective: Slides uniform; use of arrow controlled, concise. 

2 . 

2, Less effective: Initial impact on audience was colder, more 

reserved than previous speakers; was there an air 
of de fensiveness ? 

3. I found uut later that many of the participants had received 
the manual earlier through the mail. The highly technical 
nature of the data "turned off" some which may account for 
the general coolness of response sensed by this appraisal. 
Would a time for questions about it have been appropriate, 
or a need to use it as the reference it seems to be? 



11:00 a.m. , Thursday — Dr. Hillman and pane. 1 

A, Although the panelists seemed to frequently move away from the 
problem cares, Dr. Hillman forced them back to complete the task. 
Smooth and orderly would not be appropriate terms to apply, but 
the lack did not distract from this session. 

B, Evaluator not qualified to comment. 

C, 1. Effective; Hillman von the audience immediately by siding vlth 

them "against" the panel of experts. His humor as 
directed sarcas tically toward the panel was werl 
received. V.hen the panelists drew conclusions with- 
out thinking aloud for the audience to hear them 
progressing, Hillman forced them to talk their 
thoughts . 

2. Less effective: When the panelists 1 natural interest, in each 

other’s work (as contrasted with the work of the 
participants in the course^ was generated, they 
tended to speak to each other and move away from 
the basic task. 

3. 1 wondered at first at the threat which this situation might 
create with the panelists. However, Hi liman started with 

Dr. Hai'ker, one of his colleagues, and opened the way for the 
others via this. It also warred uo the audience toward Dr. 
Harker because he so obviously knew what he was talking about 
and knew when to stop; in other words, both hi liman and Harker 
were sensitive to the situation. 
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4:00, Thursday — Laboratory demonstrations 



Folse : 

A. because this was to be used the following day, it seemed 
almost illogical to consider after the sessions just held. 

C. However, Dr. Folse's presentation was direct, oriented to the 
"field” use of the equipment, 

Hougie-Rapapor t: 

A. No formal presentation made while I was present. 

C. Two persons seemed to dominate the entire time by their questions 
which tray have been relevant to others, but this ended up being 
a conversation between four persons rather than involving most 
of the group. Mien speci fir. cases were rcenti oned, however, 
all perked up. See notes on 2:30 session re this. 

Marker ; 

A, Tignrly organized and smoothly presented despite allowance for 
multiple questioning interruption* 

C. Mimeu details were available for later reference; delightfully 
warm response to questions which generated more from many. 

There was an assumption again about the ability of the parti- 
cipants to do what the presenter was skilled in doing, i.e, , 
find 16 in the slide; speakers must think about the abilities 
of the viewers to read the data as easily as they do. 



Session r fl: 9 a.m. , Friday — Dr, Gardner 

C. 1. Effective: Conversational rather than dynamic. 

2. IjG s s effective: I again wondered abt-ut the ability of the 

viewers to read graphs as readily as those do 
who are steeped in research. Presently try own 
skills have been in rather constant use re this, 
but I was not abJc to follow the speaker as 
rapidly ns he seemed to think l should be able 
to. Spoke overtime. 
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9:45, Friday — Pane] on Diagnosis 

Bruce: Dynamic, well organized. 

Figley: Monotone Dot exhibited great care in reading his slides that 

all could follow. 

Wein right: Careful description of slides a 1 though seemed quite self- 

conscious in first appearance at least. 

Folse: Excellent explanation regarding what to hear and see; the 

coordinated use of sight and sound impressions seemed to cap- 
ture the imagination of the audience (novelty?) 

Butler: Tic questioning during Ms presentation forced audience alert- 

ness rather than the usual passive intake of data; although 
the drawing for first presentation developed the ideas in 
logical sequence, the second drawing simply took time to 
create without forcing thought to follow with it. I suggest 
the use of overhead projector transparencies for such thought 
development with arrows, etc. drawn cn by the presenter when 
appropriate and/or use of overlays to add to the sequencing. 

3. Several questions from the audience sought criteria for 

decision-making, which seemed to show a felt-need on their 
part that perhaps the presentations were not making satis- 
factorily. Tiiis was too long a session 9-11:30 without break l 
The moderator (Dr. Finch) did an excellent job in providing 
a rationale for each aspect of the presentation by which a 
unity was achieved that vas missing in the previous day*s 
presentations. The end of the session again was reduced to 
interaction between the participants; perhaps they should 
have some lunches or dinners together to allow for this and 
not distract from the learning of the participants. I re- 
cognise the value of some of the dialogue as it is "overheard ," 
but it quickly degenerates to matters of primary concern to 
the researchers rather than the practitioners. 



(Although I attended the 11:30 session, I have no significant notes re- 
garding it since I had to leave early to attend a 12:30 class. Then 
I was able to return for the small groi p discussions at 3:50.) 



Session IV: 3 : 50 p.m, , Friday — Dr. Hillman, chairman — Case Study 

Discussions 

A. The case studies forced the focus to an organl zat ion , although 

several comments by the participants regarding the cases night be 
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relevant: Case # (1)> t>. 1: Question -- the first question is 

artificial; it obviously wouldn't be such a straightforward case 
to be included for consider* tion ! On rage 3, the third ^uestior, 
tells the expected answer; page 4, none of the participants knew 
what the Hardisty test meant — was it included to distract? Tf 
significant, it should have been treated in the earlier sessions, 

C, The moderator (even in the obvious presence of this "outside. 11 ob- 
server) created an atmosphere of relevance with a sense of author- 
ity cn Ills part (although he denied that he fit such a category). 

He did his thinking aloud as he took a turn on each sheet of each 
case for all to follow; several times to digress to his "biases" 
but recognized them as such; it seemed difficult to get the parti- 
cipants into the "guessing" gamc--that is, they seemed hesitant 
to put their thinking on the line, but Hartman gradually eased them 
into an increase ot this; this was accomplished in part by keeping 
the participants from conrritting tlutrselvcs to gross errors by 
leading them Jo'‘Ti the straight path with gentle nudges. 
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"HEMOKHHACE AND TliKOMBOSIE COl'KSE EVAUTATION" 
May 23-24 , 1968 



May l preface my comments v/it-i the statement that this Is the first 
of these sessions I have attendee! or frit remotely qualified to attend. 
Therefore, I hope my comments arc not just a statement of the obvious, 
irrelevant, too specific or too impractical. In fact, I may have missed 
the poirt entirely. However, we were asked to put down our personal 
ideas, so I did. 

The tvjo-day course, I feel, is good because O') Most of the re- 
gistrants are probably from out of tovm — anything less would hardly 
be worth it, ( 2 ) \ou nave a chance to "sleep on it 11 and therefore get 
points cleared up the second day which you weren*t sure about the first. 
Some determining factors in its o f fcctiveness are going to be the hetero- 
geneousness of the audience and the scope and depth of material covered. 
The scope and depth of the material to be covered is determined by the 
audience you arc trying to reach. I feel this was done very well on 
the whole for this course. 'Hie course was well titled and the subject 
matter for the lectures or sessions quite explicit. Sending the manual 
out ahead of tine was an excellent idea since it cut down lecture time 
needed for review, nomenclature, theory, etc. 

The clinical orientation was good since the audience was probably 
mostly out-of- town clinicians and technicians who expected to take in- 
formation home from this course that they could use to improve what 
they alieady had. Here T. might mention I feel panel discussions of 
problem cases an excellent and practical teaching device. It brings 
out the practical problems much more effectively and makes audience 
participation easier. 

Having guest speakers from other parts of the country is a must. 

It Is tremendously valuable from the standpoint of giving different 
experiences, viewpoints and areas of emphasis. 

The lectures on the whole were good. In only one or two cases 
I thought an excess of time and effort was spent just proving a point 
and could have been done more succinctly. The questions and dis- 
cussion from the audience is a good thing; however, the size and hotero- 
geneousness of the audience must bn considered. 

Small discussion groups are good, but here again, the less hetero- 
geneous the group, the better. Maybe this could be reduced without 
losing the M fluidl*y," 
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My personal feeling is that methodology should have been better 
covered, but I realize I am spanking as a technologist* The tests 
to be used and the reasons for using then were mentioned throughout; 
but what was essentially over 1 ooluid— and this may be unique to coagu- 
lation — is what I will call "hazard of the laboratory," The primary 
cause of this is that nest of the coagulation tests are biological 
assays — indirect measurement of activity from which quantity is often 
inferred. Consequently, the chances for artifactual influence are 
many and I think dangerous if rot recognized, I presume the laboratory 
demonstrations were meant to take care of this, and this is fine for 
something specific such as the platelet count and doppler device but 
general clotting techniques is much too broad a subject to be covered 
here* Better to discuss the various tests, their limitations, and 
the hazards inherent in each and how to recognize them, (See 
recommendations, ) 

In order that more people get more out of the course, I would 
suggest the following: 

(1) At the beginning of the sessions the registrant could, if he 
wished, write on a provided slip of paper any tiling he especially wanted 
to know or was interested in. These could be placed in a box and 
collected. It may hilp in the forming of small discussion groups end 
give an idea of what the registrants want. 

(2) Questions following a lecture and in discussion groups are 
very good, but sometimes the best questions don’t get asked because some 
people are sinply too shy or feel their question might be stupid, irrele- 
vant, or not of general interest. Would it be possible for questions 

to be written with name of lecturer and have a panel answer these later? 
This is probably impractical Lut I do think there is a problem here. 

(3) It would have helped me if the lecturers could have passed 
out a mimeographed summary of their talk with pertinent slides thereon. 
Maybe J am not a good "adult" student and am asking to be "spoon fed", 
but since the course in concentrated, retention becomes a problem. In 
trying to take notes and copy slides, I was in danger of missing part 
of what he was saying. With the mimeographed sheet I could have merely 
underlined and written questions in the margin; then what I had to take 
heme would be more organized and meaningful than my present notes are, 

(4) For the technicians I vould recommend a session in which new 
techniques or test's are discussed. This discussion would include such 
things as equipment necessary, case in performance, reproducibility, 
specificity, pitfalls (how to recognize and avoid them), and circum- 
stances in which the test would be of value. Actual "recipes" would 
not be necessary, but a rireographed list of references would be 
great. 



On the whole , I was very nuch impressed with the course, 
certain ly s tinul at ing and 7 feel I got a great deal out of it. 



It was 



37 



Mary Willemln 
Coagulation Laboratory 
Children's Orthopedic Hospital 
and Medical Center 
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"HEMORRHAGE AMD THROMBOSIS COURSE F. VALUATION" 

My aim m enrolling. in the program was to obtain current infor- 
mation on the various parameters of hemostasis and a fremev/ork for the 
rational approach to the diagnosis and treatment of hemorrhagic and 
thromboembolic disorders. My hopes were realised. 

The thought and planning that went into the course was obvious 
from the first lecturer on. The overall approach to the problems of 
clotting and then thrombotic disorders w*as logical, sequential, and 
complimentary. Its organization and presentation of 11 take home" know- 
ledge exceeded the two A.C.P. courses I have attended. 

The following contributed strongly to the effectiveness of the 
course: 

A, T he excellent Hemostasis Manual was succinct with appropriate, 
pertinent diagrams and charts. It will be very useful long after 
the course. 

B, The hand-out case histories made available for the audience were 
thought provoking. This again had considerable ’’take home" value. 
(They should have been handed out on the first day so that those 
interested could have worked on them that evening. I noted several 
people working on them during the second day of the course while 
the lectures were going on.) 

C, The presentation of several problem cases to an expert panel was 
one of the high points of the session. Dr. Hillman’s presentation 
and moderation was excellent. He was a perfect foil in his 
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persistent questioning, of the expert panel over its approach to the 
problems presented* (More time should have Veen allotted for this 
so that the third case could have been co.v,. idci ed, Would it not 
have been more appropriate to have this session toward the end of 
the course? For example, one case had elements of intravascular 
clotting and yet Dr. Rapnport did not cover this topic until the 
second day,) 

D, The final small group sessions sound life a\ excellent idea to 
allow one to clear up questions on a pei fion-tu -person basis* ( I 
was, unfortunately, unable to attend,) 

E, The choice of guest faculty was excellent* It allowed for a breadth 
of ideas and approaches that is unlikely if all the faculty are 
from the same school. This was demons traU d repeatedly during the 
course * 

F, The excellent breakdown of much complex information on anticoagu- 
lation by Dr, Seaman was most helpful* The handouts were of great 
aid in summarizing considerable, complex information, 

C, The choice of experts in various fielca: for example, Dr, Rapaport 

in intravascular clotting and Dr. Darker on thrombokine tics, Their 
enthusiasm while talking on these subjects was infectious, 

Minor deficiencies seemed to tie to be: 

A, Less than ideal slides in some cases, Foi instance, if all the 

slides during the first day had been consistent with the Mom a s tan is 
Manual , they would have been more readily grasped by some and would 
have aided in the repetition and reinforcement of the information 
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in the manual' 

IS, At tines, the projection of slides was very poor with regard to 
focusing and keeping up with the lecturer* Mach tine the speaker 
must wave, repeat his request for the next slide, etc*, the train 
of thought and concentration is effectively broken. 

Summary: The course had breadth, variety and cvrrentness* The 

presentation of material was generally excellent and tl * "take-home” 
material could not he improved upon. For my purposes, the course was 
all that I had hoped it would bo. The audience nay have been less 
receptive than it could have been. 

Sorry I can't be more critical! It was a good course* 

K. P. Potter, M ■ D • 
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E xample 4 

"THE pilot ltudy of the techniques used and the time 
sp fnt by physicians in continuing education" 

Surprisingly little is known about the specifics of the techniques 
used or the time spent by practicing physicians in continuing education. 
Approaches to measure each have resulted in significant distortions con- 
sequent' to the presence of outside observers. The need for such data 
exists for many reasons, including the deterninat ion of success to any 
change in behavior which may be attempted, i.e«, do M«I)» f s respond to 
any innovations in continuing medical education? 

The pui , ose of this pilot study was to obtain data on how one par- 
ticular group of physicians ’’continues to learn." A basic premise was 
that physicians are continuing to learn even though they may not believe 
enough. The method employed ir. this investigation was that of a time- 
and-mot i^n study. For the purposes of the study, physicians 1 behaviors 
were divided into two main categories — that related to behaviors directly 
or indirectly contributing to physician overall "medical learning," and 
all other forms of behavior--cxamining patients, driving a car, boating, 
skiing, etc. The tern "medical learning" is intended to be broad, such 
as socio-economic items, political items related to medicine, but obvi- 
ously, not all-inclusive. The "medical learning" behavior was subdivi- 
ded into several categories: 1) Ver bal behav ior-- face- to- face confron- 

tation with professionals, telephone conversations with professionals, 
conversations with pharnaceut ical representa t ives , attending rounds, 
lectures, conferences, elc. ; 2) Read 1 ng bchav i or — medical Journals, 
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both subscription and throwaway, text books, magazines and newspaper 
pamphlets, books, reports, letters, and consultation reports, etc.; 

3) rn anua 1 behavior- -practicing to perfect a skill related to medicine, 
trying out a new device, listening f ;o heart beats, etc.; 4) al l othe r 
learning behavior — a written description of each. A diary record of the 
various learning activities became a matter of simply checking one of 
the distinguishable categories on a 3" by 5" simplified card form at 
random times thro.^bout the day. To sample their activities at random 
times throughout the day, a portable random alarm mechanism--smaller 
than a pack of cigaret t.es--vas worn by those participating in the study. 
The random alarm module (RAM) is a self-contained portable, battery-oper- 
ated device which generates alarm signals at pseudo- random intervals. 

A small piece speaker plugged into the output jack of the RAM has a 
period of high frequency whistle (approximately 10 seconds) at random 
intervals. The basic intervals selected by the factory electronic com- 
ponents are varied during field use averaging 1-3 hours. The user has 
a small thumb-wheel which is moved after the output signal has cause-* 
him to record the necessary data. Moving this wheel sets an indeter- 
minate time into the RAM before the next output signal occurs. 

The sampK for this pilot study was comprised of five physicians 
on the staff of the Virginia Mason Clinic, which is recognized as an 
exemplary representative group of specialty practice. Not only would 
tills method permit documentation of what physicians actually were doing 
but also resampling after introduction of some new approach to continuing 
medical education would let all concerned know whether it had any impact. 
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During this pilot study, the five participants recorded the acti- 
vities they were involved in at the tine the random alarm sounded. A 
total of 123 responses were recorded on 15 cards. These contained 25 
references to continuing education activities or 20 per cent of th.* 
responses were so designated. Broken down, these activities includ'd: 
house staff contact, 1; conference, 2; telephone to doctor, 1; talking 
to doctor, 2; talking with the nurse, 3; attending lecture, 2\ teaching 
rounds, 7; reading behavior, throwaways, 1; text book, l; journal arti- 
cle, 2. About 3 per cent of the responses were related to reading 
behavior; about 17 per cent to veroal behavior. Interestingly, no con- 
tact with detail representatives was recorded. No mechanical behaviors 
or other types of learning activity were recorded during this brief 
sampling period. The sample of physician learning behaviors was not 
large enough to make comparison between the sub-categories under reading 
behaviors or verbal behaviors. These findings suggest that a relatively 
small proportion of the medical learning activities of physicians are, 

In fret, those in a category of reading behaviors. 

Tills pilot study did yield information to support the postulates 
that 1) this method of assesslrg the learning behavior of physicians 
is, in fact, a feasible method; 2) the RAM device, even though distract- 
ing the physician somewhat frot.i his work, is av efficient and reliable 
meins of piovliing a random sample of physician activities, assuming 
that the physicians will conscientiously record their activities at the 
indicated random time intervals; 3) physicians arc able to distinguish 
between verbal learning behaviors and reading learning behaviors; and 



o 

ERIC 



Ho 



64 



4) hypotheses concerning the learning habits of physicians can be set 
forth to be tested in a larger study comprised of an appropriate sample 
of physicians over a sufficient length of time for statistical analysis. 

Based upon the findings from this pilot study , the investigators 
believe these data are- encouraging enough to warrant a research project 
that will yield valuable information that has not been available here- 
tofore* The implications from the results of a larger study could yield 
significant information about the learning habits of practicing physi- 
cians that conceivably influence the nature of continuing medical 
education programs. Preliminary discussions have been held with a group 
of rural practitioners; unfortunately, despite the presence of a moni- 
toring senior medical student in their respective office settings, 
their failure to wear and to record appropriately occurred with such 
frequency as to make this pilot study totally incomplete. So, too, 
discussions continue with a group of house staff in the University r s 
residency program seeking to involve them in a comparable approach. 
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In summary, the examples presented illustrate, some of the methods 
used to evaluate continuing medical education courses. They have 
included pre- and post-knowledge tests, attitude measures, rating scales 
questionnaires, and observations by health and educational specialists. 

A variety of techniques should be available to gather different kinds 
of information to assist in making various types of decisions. 

More complete write-ups of selected program evaluations are found 
in the Appendix. 
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OBJECTIVE 5: Selective continuing education prograns as a model for 

investigating why some physicians choose to participate 
in these programs and vhy some choose not to participate, 

The two programs agreed upon by the Contracting Officer and the 
Project Director were ''Urology for Kon-Urologists" and "Early Detection 
of Disease." Each participant in these courses was given an information 
sheet in order to gather demographic data as well as to ascertain vhy 
physicians attended, both through the determination of one primary rea- 
son from a set of five prescribed reasons and through an open-ended 
question, "In my own words, my primary reason for attending was...." 
Self-return post cards were sent to a random sample of physicians who 
were invited, but who did not attend, requesting 1) demographic data; 

2) primary reasons for not attending from a set of prescribed reasons, 
one of which was "Other"; 3) the physician's opinion as to whether or 
not he had kupt updated regarding most recent research related to his 
practice; and 4) what topics would prompt the physicians to attend. 

For the course, "Urology for tton-Urologis ts , " all the participants 
were also asked to estimate through a four-point rating scale the use- 
fulness that each ascribed to current medical literature, specifically 
in the field of urology. On self-return post cards sent to a random 
sample (n*297) of those physicians who had been invited hut who did not 
attend, a request was made for the physician's opinion about whether he 
had kept "updated" regarding most recent advances related to his prac- 
tice, and what topics would prompt him to attend a short course* 

Various responses were made to the statement, "In ny own words, 
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my primary reason for attending was...." The following are representa- 
tive of the responses: 

to be sure I’m with it. 

... to make certain X was giving the best up-to-date 
care and diagnosis. 

... to keep abreast, hence adding to one's confidence. 

... a check on my own mett ods with an eye for updating, 

... as a refresher course. 

to get the specialist's view on recent developments. 

The reasons for attending the conference are given in Table 6. 

The most frequent response, "not sufficiently updated," was given by 
48 per cent of the participants. 

Of those attending the conference, 58 per cent viewed current 
medical literature as "of little use 11 as a resource for acquiring use- 
ful information in the field of urology, nonetheless. Table 7 shows 
33 per cent saw medical literature as "reasonably helpful," and only 
6 per cent viewed it as "extremely helpful." 

Returns from the non-participating physicians were received from 
53 per cent of the sample. Reasons this group gave for not attending 
the conference are presented in Table 8, the most frequent reason being 
"could not f-*oe myself from patient obligation, 1 ' followed by "program 
lacked relevance for me." 

Age of physician nay be a factor in attendance since the average 
age of non-participants was 49 years compared to 42 years for those 
who attended. In terns of their responses, there vas no significant 
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difference between attendees and non-attendees regarding whether they 
were in g rou P or solo practice. Of the non-port icipating physicians 
responding, 76 per cent felt they were adequately updated regarding 
most medical research relating to their practice. Only / per cent felt 
they were not. Seventeen per cent were not sure. This is in notable 
contrast to the replies of those who attended, of whom only 48 par cent 
felt they were sufficiently updated (p<.001). 

"Early Detection of Disease 1 ' was the continuing medical education 
topic of the program for the Eleventh Annual Meeting of the Washington 
Academy of General Pract it loner s, planned jointly with this association, 
the Division of Continuing Medical Education, and the Department of 
Medicine of the University of Washington. Instructors from the Depart- 
ment of uedicine were selected to make presentations on topics which 
would be pertinent to general practitioners. Over 200 physicians in 
general practice from the Northwest attended this conference, most of 
whom were from the state of Washington. Most of the participants and 
a random sample of non-par t icipnnts were queried regarding reasons for 
attending and not attending this course in a way similar to that dis- 
cussed above. The participants were asked to check one of three rea- 
sons for attending and also, in their own w-ords, give their primary 
reasons. These data arc found in Tables 9 and 10. Nearly two-thirds 
indicated that the primary reason for attending was because "material 
would be of help in ny practice.' 1 It Is Interesting to noce that 29 per 
cent gave, as a primary reason for attending, "a chance both to learn 
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and to get away," Practicality and a break from practice seemed to rate 
high as primary reasons for attending. 

When given an opportunity to write in their own words why they 
attended, the most frequent category (36 per cent) was a rather general 
and vague statement about wanting to learn and/or continue their educa- 
tion, Apparently, a large percentage of those attending found it dif- 
ficult to state specifically what they hoped to get out of the confer- 
ence, The second most frequent response (18 per cent) was "to keep 
updated." This, combined with "review" or "refresher" accounted for 
30 per cent of the responses. The fact that 13 per cent gave program 
topics as a primary reason for attending is of particular interest. 

The attractiveness of session titles may well be an important activating 
factor for some. 

In Table 11, it is seen that the most frequent reason (52 per cent) 
for not attending the conference was "patient obligation." This was 
followed by "personal or family" reason (24 per cent). Geographical 
location (at least for those who responded) was not a hindrance to 
their attendance. One must be careful in drawing generalizations from 
these data, however, since the replies represent only a small percent- 
age of those who did not attend. 

The findings from the survey of continuing medical education for 
physicians in the state of Washington (see Appendix) are quite consist- 
ent with what we found in surveying both participants and non-partici- 
pants In these two courses. Nearly half of the physicians in this 
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survey said that the greatest deterrent to their attending courses is 
time; and it makes difference if the course is in-state or out-of- 
9 tat?* 

Of the critical comments* 39 per cent were that the courses were 
not practical. The Washington State physicians ranked specialty group 
meetings highest in attendance followed by one to two* three-day sympo- 
siums, with extensive postgraduate courses attended least. Specialty 
group meetings were ranked highest as to c.erall effectiveness* fol- 
lowed again by one- to three-day symposiums* whereas the county medical 
society meetings were rated as least effective. The survey further 
showed that one out of three physicians had suggestions as to subjects 
which have not received sufficient attention in continuing medical 
education programs and that the practical aspects of practicing medi- 
cine were thought to be slighted most. 

In summary, participants and non-participants of two selected 
courses were studied in search of reasons why physicians attend and why 
they do not attend continuing medical education courses. Most frequent 
reasons given for attending were feelings of not being sufficiently 
updated, the content was seen as material that would be of help in 
practice, and for educational experience in general. A large number 
expressed that it gave them a chance to learn and to get away. An 
analysis of the open-ended responses showed an expressed desire for 
assurance, confidence, and a sense of practicing good medicine. 

These data suggest that those who attend courses have a greater 
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sense of Inadequacy while those who do not attend apparently tend to 
feel they keep updated in terms of their ovrn practice* 

Both surveys seem to substantiate the fact that geographical loca- 
tion is not a very significant factor in whether physicians attend or 
do not attend continuing education courses* However, time and patient 
obligation arc prime deterrents to participating. The practicality and/ 
or relevance of course content to physician practice appear to be one 
of the most important factors motivating attendance at continuing 
medical education courses. 

Of the reasons given for not attending, the major ones were rele- 
vance, whether or not the program lacked relevance, and the fact that 
many physicians could not free themselves from their patient obligations. 
With 63 per cent of the physicians who responded citing these two rea- 
sons for not attending, it would be well for those planning continuing 
medical education courses in the future to reconsider the time when 
such courses are cunductcd, as well as to plan courses more relevant 
to physician needs. 
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TABLE 6 



REASONS GIVEN BY PHYSICIANS FOR ATTENDING 
"UROLOGY FOR NON-UROLOGISTS" COURSE 



N- 33 



Reason 


Per Cent 


1, 


Not sufficiently updated. 


48% 


2. 


Increasing number of patients with 
problems in this area, 


12% 


3. 


Chance to learn and get away. 


15% 


4. 


To be familiar with latest research. 


15% 


5. 


Other. 


9% 










TABLE 7 



COURSE-ATTENDING PHYSICIANS' RATINGS OF CURRENT 
MEDICAL LITERATURE AS A RESOURCE FOR CULLING OUT 
USEFUL INFORMATION IN UROLOGY 



Useful Index 



Per 

Cent 



1, 


Extremely helpful (most 
important source I use to 






"update” my practice). 


6% 


2. 


Reasonably helpful (I often 






use this source) - 


33% 


3. 


Of little use to me 






(occasionally helpful). 


58% 




Of no use to me (an 






impossible task). 


— 


5* 


No comment 


3% 







TABLE 8 



REASONS GIVEN BY PHYSICIANS FOR NOT ATTENDING 
’’UROLOGY FOR NON-UROLOGISTS" COURSE 



1 . 

2 . 

3 . 

4 . 

5. 

6 . 

7 . 



Reason 


Per Cent 


Could not free myself from patient 
obligation - 


33% 


Program lacked relevance for me. 


30% 


Unsatisfactory previous experience with short 
course. 


1% 


Inconvenient location. 


7% 


Personal or family reasons. 


12% 


Too expensive. 


5% 


Other. 


12% 







TABLE 9 



PRIMARY REASON GIVEN FOR ATTENDING WAGP CONFERENCE 



Reason 


Per Cunt Response 


1. 


I felt material would be of help 
in jay practice* 


65% 


2, 


chance both to learn and to 
"get away. 11 


29% 


3* 


I wanted to find out never methods 
in diagnosis. 


6% 
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TABLE 10 



PRIMARY REASON FOR ATTENDING BASED ON OPEN-ENDED QUESTION 



Ca tegor i es 


Number 


Per Cent 


1. 


Educatic.i&l (General) 


55 


, 36% 


2. 


New material — keep updated 


27 


18% 


3. 


Program topics 


"0 


13% 


4. 


Review and/or refresher 


18 


12% 


5. 


Professional organization — 
soci alizing 


9 


6% 


6* 


Practical material 


8 


5% 


7. 


Diversion 


6 


4% 


8. 


Professional credit 


5 


3% 


9. 


Other (proximity, tax deduction, 
"pearls") 


4 


3% 




Total 


152 


100% 
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TABLE 11 



REASONS GIVEN FOR NOT ATTENDING WACP CONFERENCE 
(N=52) 



Reason 


Per Cent 


1. 


Patient obligation 


52% 


2. 


Programs lacked relevance 


2% 


3. 


Unsatisfactory previous experience 
with short courses 


0% 


4. 


Inconvenient location 


6% 


5. 


Personal or family reasons 


24% 


6. 


Too expensive 


0% 


7. 


Other 


16% 
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OBJECTIVE 6: Develop and field test evaluation procedures of clini- 

cal communication systems, e.g., hospital patient dis- 
charge summary sent to local medical doctors, telephone 
conversation and/or correspondence between hospitals 
and referring physicians, etc. 

An initial appraisal of the impact of consultants 1 reports in 
our region had already been conducted at the time of the initiation of 
this Contract. A copy of the report focusing on reaction of the reci- 
pient is included; under the aegis of the Contract, a follow-up study 
focusing on content analysis (also included) was conducted. As can be 
noted from the data presented, in the absence of the transmittal of 
additional information, the traditional consultant's letter "summary 11 
proved remarkably effective in bringing to the local community physi- 
cian between 75 and 90 per cent of the totality of information he 
needed to act appropriately with this patient. While some deficits 
did exist, it is fair to conclude that any deleterious effect on patient 
care would more likely arise ft/otn non-application of the information ns 
opposed to its non-t ransmit tal . The inference was drawn that on theo- 
retical grounds the discharge summary or consultant’s report ought have 
signigicant impact as a vehicle for continuing medical education. The 
Department of Medicine at the University of Washington has boon approa- 
ched and is currently considering implementing some of the recommenda- 
tions offered. 

Yet another clinical communication system has received preliminary 
consideration, After some four years of discussion and with a multitude 
of support techniques, "Medical Radio Conferences" were introduced to 
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the Pacific Northwest in September, 1968. In collaboration with the 
University of California (San Francisco), the Project Director arranged 
for the production of some 18 hour-long local panel discussions on a 
variety of medical topics. When these were added to those generated 
in San Francisco and the series broadcast, the potential existed for 
broad reception throughout Washington and into parts of Alaska. More- 
over, an additional series of conference calls was held with the 46 
physicians in Southeast Alaska — via seven location points following 
their local choice and review of relevant topics. 

Despite only a year’s experience, it was deemed desirable to try 
to gather some simple measures of reception and potential receptivily. 
Both interviews and questionnaire post cards were used, neither o£ which 
was financed by the Contract. Replies were secured from 65 of the 120 
hospital staff throughout the state — some 30 of which were, unfortu- 
nately, outside the reception area. Personal interviews were conducted 
at some 20 institutions in thu t Sound area and in Spokane. The 
questionnaire returns can be summarized as follows: 

Fourteen hospitals have already installed FH radios and another 14 
are in the process of doing so. Nineteen staffs are known to have lis- 
tened to the program series ort at lea^t one occasion as a group; the 
total number of individuals involved is not known. Twenty-five of the 
hospitals look forward to involving nurses and other allied health work- 
ers and 13 indicate they would be willing to pay $100 a year for sub- 
scription costs. In view of the limited promotional campaign involved, 
this is somewhat of a more favorable response than anticipated. 

o 
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OBJECTIVE 7: Evaluate the recently initiated seminar program conducted 

by representatives from the Schools of Medicine, Archi- 
tecture, Education, and Communication Arts for those 
faculty members participating in continuing medical edu- 
cation programs. The purpose of this seminar program is 
to improve understanding of the learning processes and 
to improve teaching skills. 

As noted in the previous correspondence to Mr. Norman E. Tucker 
(May 5, 1968), shortly after the initiation of the contract, the semi- 
nar program referred to in Objective 7 was discontinued. Preliminary 
appraisal of the previous seminars was forwarded with above-mentioned 
correspondence. 
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OBJECTIVE 8: Select two additional continuing education activities 

conducted by the School of Medicine for evaluation 
purposes. Activities to be selected may include: 
relevance of medical student appraisal programs for 
practitioners education; relevance techniques cur- 
rently employed to assess needs of residency training 
programs; and methods to improve the quality and 
impact of medical journals. 

Even though the original intent was to select two programs to 
evaluate, more than two were evaluated under this contract. Evaluation 
reports are found in the Appendix for the following programs: 

1. Urology for NonMJrologis ts 

2, Computer-Assisted Instruction 

3, Early Detection of Disease 

4. Central Washington Project (Yakima) 

A study of internship advisors is also included; so, too, are four 
graphic representations of a Semantic Differential inventory of students 
and faculty attitudes and their surprising congruences. Both have impli- 
cations for continuing medical education. 

In addition to these, the contract staff consulted in the develop- 
ment of measuring instruments, objectives, and evaluation designs with 
the Washington/Alaska Regional Medical Program staff and project direc- 
tors for other projects including: 

1. Sub-Regional Coronary Care Educational Programs 

2, Postgraduate Precep torships for Physicians 

3. Guest Residency Program 

4, An Educational Program in Finding and Treating Lung Failure 
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In conjunction with the pursuit of Objective 8, preliminary studies 
have been initiated regarding the impact of medical journals* The 
relevant usefulness of current medical literature as detected from our 
"Early Detection oi DiseasV (Olympia) and "Urology for Non-Urologists' 1 
studies (Objective 5) is reported in Tables 12 and 13. The general 
practitioners found specialty journals least helpful as a resource for 
culling out information. General journals and "free" journals are seen 
as extremely useful by 24 per cent and 14 per cent , respectively. Spe- 
cialty journals were seen as "of no use to me" by 12 per cent of those 
attending the Olympia conference. 

The Project Director has extended a study begun several years ago 
into the characteristics of pediatric specialty journals and their con- 
tinuing medical education relevance. Two preliminary reports available 
to date (attached) permit inferences to be dravn--agai n pointing up the 
cont radic t ions which evolve when a given task is contrived to solve two 
or note almost mutually exclusive problems* Phrased differently, how 
can a publication looked to by readers for clear, concise updating of 
relevancy, tf -. M ver such information when It is prepared by non-^i li~ 
tioners with content reflecting either their research interests or 
their personal publication needs? cl though v:e have no immediate rec- 
ommendation to resolve the problo , : .v ire proceeding first to attempt 
to define its extent and then to evolve alternative appro, dies to 
resolution. £s a consequence, additional studies are being planned on 
this particular topic. 
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TABLE 12 



PER CENT OF GENERAL PRACTITIONERS WHO PERCEIVE 
CURRENT MEDICAL LITERATURE AS A HELPFUL RESOURCE 
FOR CULLING OUT USEi’JL INFORMATION 



Type of Journal 


Extremely 

Helpful 


. _ ■ -i 

Reasonably 

Helpful 


Of Little 
Use To Me 


1 Of No Use 
1 To Me 


General Journals 

(i . e. y JAMA(GP), NEJM) 


24% 


62% 


14% 


0% 


Specialty Journals 

(i . e. , Ann. Int . Med. , 
J • of Peds . , etc. ) 


6% 


33% 


44% 


12% 


"Free 11 Journals 

(1 .e. , Med. Mid . News, 
Mod . Med . , e tc . ) 


14% 


57% 


26% 


3% 



■ TABLE 13 

CURRENT MEDICAL LITERATURE AS A RESOURCE 
FOR CULLING OUT USEFUL INFORMATION IN UROLOGY 



Useful Index 


Per Cent 


1, 


Extremely helpful (most important 
source I use to ,, update M tny practice. 


6% 


2. 


Reasonably helpful (I often use this 
source) 


33% 


3. 


Of little use to me (occasionally helpful). 


58% 


4. 


Of no use to rre (an Impossible task). 





5, 


No comment. 


n 
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Preliminary Report: submitted to 

American Journal of Diseases of Children 



PUHLi ISH , P ERI SH — OR POOP OUT 

Piqued by curiosity, I recently updated a Btudy begun some ton 
years ago* The object: to look at our pediatric journals » to see 

what they arc saying* who is saying it, how long they take to oey 
It, etc. The subject: Pediatrics , The Journal of Pediatrics * and 
T he American Journal of Diseases of Child ren? (at inception. Clinical 
Pediatrics was not yet on the scene)* Tho method: Simply cat up 

criteria, count and record data by specific journal \ thon compute 

0 

averages, percentages, and so on. The years covered: *58, *59, and 

*60 versus *67 and ’68, Nothing too fancy--save for application to 
the ’recent set of Foinstoln's memorable categorization on " Non-human - 
non-disoaso n papors--ono other way of grappling v;lth the issue of 
relevance of journal content to problems of practice. 

Each of those three journals provides nearly 1,000 pages of 

textual material every aix months* Figuring 700 to 1,000 words poi 

page, the average reader proceeding at 300 words per minute for 

reading--not necessarily for "studying 11 or "undorstanding'^-would 

take approximately 160 hours every six months or about six hours a 

week to consume their respective contents. Even thon, he would 

have missed almost 65 per cent of tho pediatric articles abstracts 

by the Yearbook of Pediatrics- -and purported to be important. At. 

wora he to add then, he would encompass only articles immediatel) 

related to the field. Obviously, the "crunch" is here, tho sqvroe: 

is on--and it ha3 boon for a long tlr.u Some available options; 

read faster, bccomo more selective, or throw in the sponge. Not 

porticulirly attractive altorna cives ; only the second can bo con- 
' • < * 

Q sidered acceptable and oven that holds reservations. 

1 " * 




2 



Rest assured, however, that authors have yet to throv: In tho sponge t 
A 3 do Sol la Price conf irjn 3 , tho Information explosion go ho calls it--or 
is it simply a "publication inflation" --contlnuos so that output of panel's 
and journals doubles about every IS years. While tho rate remains rela- 
tivoly constant--and has for more than two centuries! — the absolute numbers 
of papers continues to burgeon goouotvically . And, so do *ho number of 
authors. Of the nora than 1,500 pediatric a?-*ticlo3 published in tho initial 
lntorval, some 30 per cent had a single author; only 15 per cent "required" 
four or noro , Over the ensuing decade, a change has occurred; now only 25 
per cent of those pediatric papers are sired by an individual; multiple 
author oh ip- -foui' or more— accounts for 21 per cent; and over tht past five 
years, iri our Pediatrics Department at the University of Washington, multiple 
authorship accounts fox 5 * 24 per cent of ell papers produced. Ono wonders why. 

Considerably more drama tic- -and conjuring up all kinds of additional 
speculation — is the fact whereas _ 1 52- 1 50-*’ CO saw some 74 per cent of articles 
spawned in tho medical schools, ’ 67- ’ Gtf saw noro than 07 por cent originate 
there. And of the 13 per cent that did not, a significant proportion stemmed 
from the ar.t-ed services — and the KIHJ By these criteria, practicing pedia- 
tricians are tho "silent majority." 
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Conclusion: Ro3 the time- squeeze for reading lead :o a retreat from 

writing? Possibly, but I doubt it. More plausible possibilities are ob- 
vious. In It that the tv,*o worlds of medicino--acudcrnIa and practlce--liko 

(?) 

Snow’s tv:o cultvies, are simply moving fui'ther* apart. Practicing piipicinns 
apparently are not recording ti;olr observations or, if they aro, they are 
having their efforts at writii • I'ojecteJ by Editorial Boards --a ofomjngly 
unlikely occur rencc . Could it be tbit virtually over pediatrician and bln 
brother iu able to secure a faculty appointment so that tho 'Vulversity 
without walls" in in reality a fact of life? Were thin option correct, it 

lub 
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v/oula portend real excl toment for the future. Should it b a another* i.o., 
that of "separatism," however* it could spell doom — and not simply for 
pediatricians and the profession* but also their patients. Careful obser- 
vation^ critical anaJyris^ ond concise* clear recording of clinical experiences* 
Is a sj.ne qua non of the practitioner who proposes to continue in the forefront 
of medicine. Although "publish or perish" may bo the scoffed-at motto of 
academe, it is ex trer.Gly relevant to hie frame of mind. And, it is perhaps 
even a more relevant concept for the practitioner unless ha is to "poop out," 

The process itsdi , as Woodford has emphasized* can bo a revitalizing 

(3) 

experience when such are clearly needed by all of us. Thus, a plea to 
practitioners for prophylaxis in practice; prepare a paper for publication-- 
promptly ! 
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PEDIATRIC PERIODICALS— PLAUSIBLY PERTINENT 

Awe-inspiring in their objectives, overwhelming in their numbers, 
scientific periodicals have obtained an almost deity status. Their 
annual content of two million papers is predicted to double within the 
coming generation, Medical journals spew o t a significant portion of 
this total ' literature , 11 

By invoking his medical journals regularly, the practitioner of 
medicine is said to learn to perform modern day miracles. By ignoring 
them, ‘.e is purported to risk the damnation of diagnostic and thera- 
peutic obsolescence. In point of fact, however, both the medical 
practitioner and the medical educator, be either a worshipper of the 
printed page or an agnostic, need to consider the question, "Does the 
physician actually learn from Ms journals?" Several reviews document 
the paucity of data relevant to this issue. This paucity prompted the 
following study of pediatric periodicals, their contents, and considera- 
tion of their relevancy to the problems and practice of this specialty. 

Today's pediatrician is cast in a unique role as he is simultane- 
ously encouraged to "care" for the whole child, the whole family, and 
in many ways the whole community in sickness and in health, physical as 
well as mental, until death (or adolescence or some other arbitrary 
criterion) do them part, Having built himself a firm foundation by con 
suming a medical school curriculum, an approved internship, and a pedi- 
atric residency experience primarily with sick hospitalized chi Mr cm, 
the pediatrician in practice deals primarily with ambulatory well 
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children and their mothers. He attempts to mninLain his specialty 
competence by a mixture of "postgraduate 1 ' endeavors. Paramount cmong 
these is devouring, absorbing, ca taboli zing , and occasionally even using 
information provided in medical journals. 

While more than 13,000 periodicals are directly or indirectly 
relevant to the medical care of his charges, the pediatrician focuses, 
to a large extent, on three monthly specialty journals and one quarterly 
periodical to keep informed of pertinent pediatric events. Respectively 
these are "Pediatr ics>" the official publication of his professional 
association, the American Academy of Pediatrics; "The Journal of Pedia- 
trics," an independent publication; and the "American Journal cf Dis- 
eases of Children," the pediatric specialty publication of the American 
Medical Associat ion. His quarterly is the "Pediatric Clinics of North 
America," a publication of Saunders Publishing Company. Of the former 
group, each issue carries reports of original scientific investigations 
as veil as reviews of clinical cases; each devoted one section tc 
worthy news releases, another to book reviews, and each has its ovn 
unique additional sections and format, i»e., editorials, diagnostic 
dilemmas, etc. 

In an effort to develop a profile of the first three journals fjr 
a comparison among then and their potential relevance to pediatric 
practitioners, a simple study has been carried out in terms ol: 

1. The total pages of textual material per volume as well 
as the t^tal number of articles. 
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2. The types of articles contained therein — i.e., ’’original" 
article versus case report. 

3. The sources of the article — academic or practice. 

4. The geographic origins — region of the country. 

5. The number of authors involved. 

6. The distribution of topics covered. 

Relevant data have been accumulated for the yerrs 1958, 1959, 
and 1960 as well as 1967 and the first half of 1968. 

a 

Results and D* . . u s s i on 

Tables 14 through 17 are in large part self-explanatory, providing 
measures of size, origins, and contents of the three leading journals. 
Some points warrant additional emphasis, however: 

1. In terms of respective objectives as stated by the three 
Journals, and by their predecessor which has now disap- 
peared from the scene (see Attachment) » it * s likely that 
all rhree have been and are being successful — in large 
part because of t h e manner in which the objectives were 
written rather than in terms of what specifics the jour- 
nals themselves have done or are attempting » r do. 

2. In terns of the broad issue of contents, the similarities 
among the three journals are far more striking than a r e 
any differences. 

3. In terns ot relevance of contone cither to medical problems 
confronting pediatricians on a day-to-day basis ( as deter- 
mined from existing sources) or to those seen as wirranting 
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mere attention by practitioners as determined by Aldrich, 
Deisher, etc., there would appear to be little evidence 
of any significant relationship. Such is understandable 
when authorship and authorship purposes are considered. 

That is, by and large, authors not only are not practi- 
tioners but they are writing in response to their own 
interests, activities, and needs as opposed to those of 
potential readers. 

4. In terms of authorship, the practicing clinician appears 
tc be going the way of the "dodo bird M and is on the road 
to extinction as he contributes less than ten per cent of 
the articles included --and this percentage is falling. So, 
too, the single author is being replaced by a "team" pos- 
sibly formed more for the purpose of promotion than for the 
purpose of publication per sc . A recent letter to the 'Jew 
England Journal of Medicine (280: 1484 June 26, 1969) con- 

firms this impression as a 40-ycur span has seen the number 
of single authors fall from 78.4 per cent in 1928 to 3.1 
per cent currently of the spavners of articles. 

3. In terms of length, however, some evidence is seen that 
parsimony may be in the wind--despi te the increment in 
authorship . 
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Summary 

Confronted with an alleged scientific and technologic explosion-- 
or is it simply a ,r publ icat ion inf lation"--the medical practitioner 
is encouraged to update himself via his professional journals. On the 
basis of a review of the origins and contents of his professional jour- 
nals, pediatricians as a class of medical practitioners are not likely 
to attach much relevance to pediatric journals 1 contents--at least in 
terms of their everyday practice or of the ''real-life health problems" 
of their patients. This discordancy is likely to increase as practi- 
tioners ccasc to serve as authors and as authors continue to write with 
an aim toward impact on an almost negligible proportion of potential 
readers. Conceivably, the "throwaway" or "controlled circulation" joui 
nals may help stem the tide of postulated professional obsolescence, 
but the words of Sir Theodore Fox, Vandercere Bush, and Peter Woodford 
regarding options to sustain "this remarkable instrument of medical 
communication" must be acted on — and soon. 
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c) Analysis of Journal Topic Coverage 

N=1 , 574 
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ATTACI1MENT 

Specific Statements of the Journals 1 Objectives 
PE DIATR ICS, 1: 4, 194 9 

’'Perhaps the chief of these is the intent ;>f the Editorial Board to pub- 

/ 

lish an outstanding journal whose articles will be practically helpful 
but which at the same time will be challenging in their scientific con- 
tent. And 1 i* sure ve all must admit to a certain pride of ownership.” 

JOUR NAL O F PMH ATRIC S, U U3 „ 19 32 

"The editors wish to explain briefly at this time the scope of the 
JOURNAL OP PEDIATRICS. It is their Intention to make it as broad and 
inclusive as the field of childhood itself. It is planned not only to 
include the clinical and investigative fields of pediatrics but iti 
addition those tangential fields of medicine and child development 
which give to pediatrics such a unique position among the medical 
sciences. Thus the fields 'f public health relations and movements, 
child psychology, socio-medical conditions, education, as veil as the 
more strictly medical branches: as for example, surgery and its sub- 
divisions, or dermatology, will be included in the material which the 
Journal will present to its readers in so far as they relate to the 
field of pediatrics.” 
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AMERICAN JOURNAL OF D ISEASES OF CHILDREN, 1; 1, 191 1 

"This new journal of ours, the AMERICAN JOURNAL OF DISEASES OF CHLhDRrV, 
however, the first number of which we herewith offer to the profession, 
is not published to compete with those already in existence . . . with 
the increasing number of physicians interested and working in the field 
of pedology, there will be more than merely a vast number of papers; 
there will be more profundity and research. These our new journal is 
to gather in, and it is to be the companion and ally of the Archives 
of Internal Medicine , It nay rarely give a mere case, still more rarely 
a prescription; it is to be a receptacle, however, of the very latest 
In the very best of American and European literatures. But extracts 
will be few in proportion. Reviews, carefully prepared by competent 
critics, will be offered from time to time. The bulk of the contents, 
however, will be original; anatomy — both embryonal and postnatal — phys- 
iology, biochemistry, nosology, and therapeutics will come to their 
own, and will be welcomed as our sources of actual progress." 

A RCHIVES OF PEDI ATRI CS, 13: 18 96 

"These facts being appreciated, the true work of the pediatric journal 
is rendered clear. It is the medium of communication between the ob- 
server and the practitioner. By observer is not meant sinply the 
laboratory worker and the pathologist, but it includes the physician 
of the hospital and dispensary. It is not limited even to these, for 
vc should not fail to appreciate Lhe invaluable experiences gained by 
the general physician* To him we allot ample space in these pages and 




1 J 



3 



urge more frequent publication of the knowledge which he gains in his 
daily contact with disease* On the whole, however, it must be admitted 



that the most important and substantial advances in knowledge result 
from the researches of special workers. The wisely conducted medical 
journal, will be, therefore, in large degree, the medium of communica- 
tion between such special workers and that great body of men who do the 
practical work of the profession. 

As the department of pediatrics cannot be restricted to a few consult- 
ants in .large cities or teachers in medical colleges, so the pediatric 
journal cannot be limited in its scope. While its best contributors 
are drawn largely from the ranks of the true specialists, the man to 
whom it should prove of the greatest actual value is the general prac- 
titioner, to whom the diseases of children are of such vital importance. 
The true aim of such a journal should be the education of the profes- 
sion at large 1 in the important department which it represents. It 
should constantly strive to place the knowledge which is being continu- 
ally accumulated by skilled observers, in such forms as to be most 
available for practical use. It has been the aim of the ARCHIVES 
throughout its whole history to attain this end, and no departure will 
be made from this policy. 
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OBJECTIVE 9: Develop a comprehensive pi on from the activities (1 through 

5 above) tor a continuing education ccuntcr for physicians. 
This plan shaLl: 

a. Do tern I no the functions to be carried out by the 
center, 

b. Set forth working procedures to be followed in 
establishing the center, 

c. Select specific areas of study and priorities of 
work to be undertaken by tlu renter, 

d. Recommend a detailed scheduling of tasks to be 
undertaken for implementation of tills plan* 

On the basis of our Interpretations and discussions of the fore- 
going as well as the existent literature, e.g., the Dyer report, the 
Millis Commission report, the Royal Commission on Medical Education, 
the Coggeshall report, etc., vc voulu endorse or propose the following 
premises as essential for any comprehensive plan of continuing educa- 
tion to be successful for physicians in the Pacific Northwest: 

From the time a physician first enrolls as a medical student in 
a medical school until the time of either his retirement or his 
demise, he ought to be maintained on one of three rosters of stu- 
dents of that school — or a consorting school: an undergraduate 

roster, a graduate and postgraduate roster, or a continuing medi- 
cal education roster. Svich would set the stage for agreement as 
to mutual obligations and those of organizations and Individuals 
as veil ns a feedback system. ► 

The student who is progressing through the above three phases will 
receive less and less specific direction as lie assumes increasing 
personal re sponsibi 1 i t y for Ids ovn’curr iculum and learning." 
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Just -■ s tii--* grade school child has mich of his curriculum spelled 
out for In,’; Jn detail, the radical student of Liu* fulurc--despl to 
the f ;at th.it. he will hive increasing opportunities to become 
invoiced with hi:; curriculum design and oontent-^can continue in 

i 

look forward to faculties of schools of nan lei tie or others in 
society proscribing large. segments of curriculum, in undergraduate 
years. But as students pr np.ivss in their training, those outside 
.a lu c a c e s diminish. The individual takes over the decision-waking 
to an over-iucreasi ng extent. And although medical school facul- 
ties are currently almost non-involved in curriculum natters for 
continuing nodical education, it is postulated they will plan an 
increasing rolc--but still a very minor component compared to that 
which will be lodged in the individual physician of the future, 

3. The past decade has seen a far closer Intcrdlgitat ion of existent 
medical schools and nodical school faculties with each other ami 
a surprising degree of standardization among them. The next 
decade will, in all likelihood, sec comparable consolidation of 
resources and centralization of many components of the educational 
process of graduate and postgraduate physicians (house officers — 
interns and residents). It is assumed that a comparable network* 
of interrelationships will shortly be formed at the continuing 
ricdlr.il educm! on level to Involve cot sir, ply Individual:: but a], so 
tlie universities, the African Medical Assoc ia 1 1 on , the A sec I it ion 
of African Medical Colleges, and specialty hoard s- -a s wvll as 



I n t c r e s t cd c o n s irv r s ! 








U . Work currently underway .it the prinary an! secondary school lavcl 
is blending the multitude of new technoln - ies of instructional 
media into the educational matrix, Work currently underway at 
the undergraduate level in medical schools is also blending in 
tiiis new instructional media but, mere significantly, it is also 
incorporating application of educational principles in the teach- 
ing- learn ing process. Tt is assumed that, shortly, this process 
will be extended to graduate and postgraduate training and then, 
in a far more significant fashion, into continuing medical educa- 
tion activities. Ta point of fact, the possibility exists that 
both the. new instructional media and the educational principles 
may become bettor integrated in continuin’ medical education 
before being adequately exploited in the other two areas. Over 
the long run, we would foresee the development of "learning pro- 
files” for individual nodical students and physicians, learning 
packages for translation into the various media*-~printod words, 
television, programmed instruction, computer-assisted instruction, 
etc. In short, refinements are well underway in the techniques 
of transmitting new information and/or skills to students and In 
predicting thoir reception and application of same. 

5. While the past generation har. seen tremendous "hypertrophy of the 

hard sciences" components of medical education and medical pricMco, 
evidence today suggests; III H the pendulum ts swing. in;; toward a 
more central position of involving (lie social sciences far more 
extensively in the overall process, I’eihaps trainees In medical 
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school In the* J 950's verc overly imprinted to perceive only bio- 
chemical bits ns opposed to psychological, social or economic bits 
being relevant to medie-il prae t ieo--or deserving of co:it Inuln:; 
nodical education efforts. I he 1970's are likely to see the psycho- 
social components assume cow.-mid, Such a transition in priorities 
is of : ..joi s \ ;-.*i j f iconc : j is- ■" . -i/Mder \ r;g a- / determination of *'n cods 
and vants 11 for the continuing medical education process. 

6. As the educational process in medical school continues to evolve 
from a highly structured* admittedly sore Limes punitive approach 
with nuch concern for failure on the part of the students to a 
more positive non- threaten in;; process, uv believe that the gradu- 
ates of s ich programs as they enter into postgraduate and continu- 
ing education activities will be far more comfortable in looking 
at their own strengths and weaknesses without fear of reprisals 
and thus participate in realistic determination of continuing medi- 
cal education needs. Little doubt exists that tviny physicians 
today find it difficult to welcome exposing themselves to the pos- 
sible onslaughts of the "golie inrat" professors of their medical 
school memories* 

With the above premises, postulates and assumptions in mini, vc 
would at first reject the concept embodied in a "Center for Continuing 
Medical l\l*u mien for Physicians/ 1 assuming the term "Center" carries 
with it the usual connotations of cent ra M at Ion of control, power, 
and decisions ro prevalent In tin modern acum-no mentality. We find 
the centralization implied an anachronism. It is totally inappropriate 
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for what, the future holds. Obviously, a function such a.s continuing 
nodical education would suffer disastrously if scon as being confined 
to a single physical facility with a region or directed by a single 
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individual and his professional societies but, equally important ly, of 
the nodical schools and adult education. Thus, in the state of Wash- 
ington, the state universities, state colleges and state community 
colleges v, 'ou Id each play a role in the process of continuing Medical 
education — in addition to the existent assoc int ions , societies, local 
hospitals as veil as the individual physician. 

Our investigations of the Regional Medical Program-University of 
Washington Vreceptorchip Program for physicians, the Yaklma-Cent ral 
Washington survey of "wants and needs, M our collaboration with Green 
River's studies, with the Washington State Medical Education and Research 
Foundation survey, and with work beginning among the Community Coordi- 
nators all serve to stress the desirability of a focus on decentralized 
and individualized programs. Moreover, our look into why physicians 
dor^t attend cour.ses--roal.-t tne and patient-load problems — suggests 
that this dcccntr.il i:*.it ion process needs continue*. Finally, even 
though never media have begun to naku possible such decentralization of 
dissemination of process, they, in fact, tend to centralize the dcci- 
s Jon-maki ng process about vli.it, when, who, and why given material ought 
be d I ssum 1 r.n tcd--ui) undesirable goal to say the least. 

Accepting this argument gives rise to the need of co«*rd i n.iLivu 
roles am missions; only if tin* ccordj native roles are insert rd via 
"participative nanagi lent" Into the definition or concept of a "center, " 
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do we feel L he real goals of continuing nodical education vill be 
achieved. Whale such a philosophy may appear to bo both naive c-n the 
one In ad and unfeasible on the other, v;e feel that it blonds with our 
findings and tint It is moat appropriate to look at the so-called 
idealized innge and out line it- -and then develop compromiser; rather 
than to compromise v 1 l!\ principle:; expressed. 

In such coordina L Ivo endeavors, four specific functional areas must 
be partitioned between individuals and organisations in terns of policy 
decisions and actual conduct: 

1. The deterninat i on of rtceds--knovledge , skills and attitudes — to 
maximize current practices as well as future activities — both 
from t lie consumer's point of view and from the provider’s* 
Obviously, even if the physician benefits maximally and there is 
no spin-off to the patient, the needs have, been inadequately 
defined. On the other hand , if only the patient is considered, 
it nay be that the physicians would become increasingly ineffec- 
tive. Self -appraf.sa 1 approaches as well as group approaches 
warrant exploration, 

2, The appropriate packaging of knowledge, skills, and attitude- 
forming techniques to respond to the stated needs vill obviously 
entail group i f f ort--bul ought have individual stimulus. 
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4. Tli 3 evaluation of vncth^r tin* transmittal of the prepared ;vK’hir.,v.; 
doss, in fact, satisfy the needs specified will call for skill:, 
of both 01 i>>nl za t i < -ns and individuals* 

In setting forth working procedures to partition responsibility fo 

those areas, several steps seem critical. They arc: 

1. More careful specification nH quantification of the existent 
problem and the premises staled above in an attempt to reach 
conceptual concordance with all the necessary individuals and 
organizations, both group and individual decisions must be 
made whether to commit efforts and resources toward refinement 
and sharpening of existent tools of continuing medical education 
or to atter.pt to create an entirely new set of organizations and 
tools for this purpose. Wo assume they will opt for the former 
alternative . 

2. Several years of discussion and involvement with the individuals 
and organizations to attempt to bring about a commitment to t he 
philosophy outlined above. Here it would be critical to involve 
nodical students at the entering level as well as those in other 
classes of the school. Of particular relevance now is involving 
faculty and house officers in moving this particular function 
(t.e, , undergraduate, postgraduate , uni continuing education) 
higher on the It professional priority list. There seems to be an 
inherent assumption that the faculty of a school of medicine run 
breathe easily once a student has grain iw-d — ro they can conci n- 
trato on other entrants. One in mediate measure in imple ■: \ nt .1 ng 
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our plan would be to have this broad objective of lifelong involve.- 
nent stated and endorse*! in the long-range plan of any nodical 
school- -I .c . , a connitnuit to equal efforts on behalf of the 
school for the throe levels of education involved. Let those 
responsible for funding foil*:: through on that detail of impLe- 
meat at Ion . 

3. Dove lo p~o nt of task forces with specified action deadlines repre- 
senting the involved organizations and individuals to focus on; 

a. Financing options. 

b. Doc ision-making processes. 

c. Communication networks. 

d. Evaluation techniques. 

4. Further expansion of tiio Washington State Medical Association- 
Washington/Alaska Regional Medical Program-University of Washington 
Council of Community Coordinators to include membership from repre- 
sentatives of various state and community colleges. Gradually, 
other allied health professionals should also be added so that 
eventually the resultant body can serve not simply as a forum but 

. also in decision-making regarding recommendations for facilities 
and equipment all of which need to be planned toward compatibility 
of he a 1 Lb sciences education. Conceivably this group will pen- 
ce Lw the value of estriM Inhirg a more structured coordln.U fve- 
administrative unit with the Council's membership serving Jn a 
"board -of- trustees capacity. 11 Wu'd *er such units organic J to: 
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a, Develop still better necd-det ermim i i on nr.il planning 
serv ice a . 

1>. Develop appropr ia to "d iuscmJr.al ion network*; 11 for television 
raj in, telephone, a*: vpII ua films, learning packages ami 
eventually compeer assisted Inal ruction. 

e. Develop nr 1 public: - J .c n library of existent programs and 
activities ;rs veil ay personnel resources. Here a parti- 
cular advantage exists in the already active Pacific north- 
vest Regional Health Sc fences Library, which fills thin 
role for printed materials. 

d. Develop new or better utilize existing evaluative skills 
and services. 

e. Develop, an consort with the College, of education and the 
various schools in our Ilea 1th Sciences Division, training 
programs to prepare discipline-based individuals to assume 
responsibilities for the management and evaluation ot con- 
tinuing education activities. Ve view such training pro- 
grams as being the most, critical ingredient of any long- 
range comp re hen si vo plan for continuing medical education— 
if it is to be truly responsive to health care needs of the 
community via implement at ion of educational principles. 

Tor example, we would call attention to the findings of our 
study oi the use of Ruo -Gun (rdi-it.i m uc globulin) ,vil prac- 
ticin’ physician ce, , r chons ion of the pat huphys lol u-.y 
involved. Obviously, .it this point In tlv’e, nu additional 
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continuing nodical cJurni Jon efforts need be undertaken to 
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transmit relevant new inf emu t!o;i although reinforcement acti- 
vities may bo called for. In its stead, wo would view Jnslrue- 
t ion in techniques of communication with patients and persuasion 
principles as indicated. Thus, need dc terminal ion tons C precede, 
the implementation plane; here, even the minimum basis of under- 
standing of educational principles permits appropriate action. 

Finally, we recognize the impact of individual personalities in the 
inplenentot ion of tills particular plan; if participants are aggressive, 
but not domineering, if they are capable of participating in two-way 
conmunicati on as opposed to monopolizing via unilateral pontif ications , 
the program can be successful- The right people in the right setting 
could get the aforementioned program into full operation wltnin a year 
provided a minimum of sustaining budget were developed, In contrast, 
the wrong people devoid of any budget would not simply prolong inter- 
minably an ever more ineffective program, but night postpone for 
decades any rectification of the problems that would onj.ue. 



SUCTION I] I 
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A researcher is confronted with r. variety of challenging problems 
in a study such as this. Also, unexpected outcomes emerge. In this 
section of the report, our attention is centered on the problems, recom- 
mendations, and implication? of the study. 

P RO BERTS ■- N’COaXTj-hlKn : 

A number of problems were encountered, some of which are listed 
as follows; 

1. The breadth and depth of the issue under consideration, the 
extent and ramifications of continuing medical education itself, 
presented an inherent problem of focus. Attempting to view the 
broad spectrum of continuing nodical education meant that para- 
meters for investigation were virtually unavailable. Neverthe- 
less, an attempt to take an overview of an area such as continu- 
ing medical education is imp *rtant in order to provide for in- 
terrelations of the several component parts so that the total 
eir.phasLs will be more unified and therefore more effective and 
efficient. 

2. Professional resistance. Because of the natural 'threat" to 
group solidarity and the professional inage that arises when 
any "outside" party questions that group's practice and behav- 
ior, the physicians as participants wore somewhat suspicious. 

Also, perhaps because of the "soft" reputation of education's 
body of knowledge, there was a fair amount of skepticism about 
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measurement procedures and the validity of the instruments. 

Four at her factors are postulated to contribute to this 
resistance to exposing personal professional inadequacies-- 
i.e., assisting in the determination of needs: 

a. Tlie past decade has seen a real shift in organizational 
p<»v r Flocks In medicine. Medical schools have burgeoned 
and assumed a tremendous control; hospitals are booming 
and capturing more and more power. County medical soci- 
eties arc 1 agging--espec ially on the educational forefront. 
In some ways — in many ways — the individual physicians no 
longer count. Further change from the good old days is 
not a 1 w ay s w e 1 c ome d . 

b. The past generation has seen an elevation of the special- 
ist and deprecation of the generalist. Part of the spe- 
cialist image i.> to be interested "in his thing. M Reten- 
tion uf tliis image was maintained by not expressing interest 
In "our thing. 1 ' 

c. Success in a professional practice — like heauLy--exis ts 

in large pail in the eyes of the beholder --in this instance, 
professional colleagues. Any public nr group declaration 
of internal inadequacies in the group could only serve to 
tar.iish such ati image with obvious real-life consequences 
to those involved 

d. For centuries the "prof css iona] u has evolved a self-account- 
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ability concept as part of hl ,; image. Opposition to this is 



increasingly obvious; r e I irensure and recertification are 
confix a*- -anti throat erring- -words . Such implies that out- 
side forces are out to assume total control of the profes- 
sional. Hence , sane reluctance to cooperate with another 
outside force. 

3. The relative novelty of research in medical education. The idea of 
studying me die: a I education was greeted wit It skepticism not simply 
by individual physicians but by medical organizations including the 
School of Medicine, the State Medical Association and many profes- 
sional societies. So, too, some of the ot ganiza t ions involved in 
this study were also novel — our Office of Research in Medical Educa- 
tion, the Washington State Education and Research Foundation, and 
The blush ingt on /Alaska Regional Medical Pro;'. rani. All of this un~ 

f am i 1 J a r 1 1 y p ro s on l v d p rob K ms . 

4. Difficulties emerged due to the absence of a) an accepted theo- 
retical construct for continuing medical education, b) a concurrence 
as to its gOv. Is by individual physicians and their professional 
organ l f /.\ L ions , c) ati extensive literature in the field, and 

d) proven techniques for its study. 

5. Diversity of program sponsorship and lack of a central contact 
point U'SuHoil in a M eatch-as-eatch -can" approach to building 
evaluation schemes. By the nature of the contract, ve were com- 
pelled to work with existing prograr.s--not having resources to 

de vo 1 e p ou r own . Only si ow 1 y did wc beg i n l o d eve 1 op i n f 1 u cn c e i >n 
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the stria lure and content of those programs v;o attempted to evaluate. 

6, In Many eases, evaluation w.s "after the fact" or "tacked on" like 
an appendage rathe'' than being an integral part of the program 
planning Itself. Nearly all the. evaluation approaches we suggested 
required son*- type of adjust? on t in the already existing program 

pi an.-. - 

7. No proven model exists for evaluation of curriculum, especially in 
the area of adult education . The contract was not designed to 
develop such a model and what curriculum evaluation schemes were 
available to us were not immediately applicable to the continuing 
med i on 1 ed i;ca t ion prog rams as eva lua ted , 

8- The acceptance of only part of our evaluation program and design 
by those responsible for the continuing education programs threat- 
ened to jeopardize entire evaluations at times. Partly as a result 
of lack of total acceptance and partly as a result of working with 
intact, groups, it is impossible to generalize many of our findings 
beyond the populations in the study. 

iJ. Seemingly lirited techniques exist for assessing physicians 1 needs 
except on an individual inventory basis. Many of these needs seem 
to be specific lo a gi’-cn specialty, a given geographic, or a socio- 
economic area. Tims, a need oi one may not be a need of another-- 
even in the same specialty because of the nature of his practice, 
tie uniqueness of his patici. population and his own priorities, 
in general, tiume individualized inventories have been applied on 
the .solf-nppiais.il basis. 




fxteptions to such approaches do exist, 
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witness SI ye , WiJ 1 :1 unison , Y> r own , White, and other audit' studies. 

In our region, audit approaches hove not yet found acceptance and, 
thus, were not available in assisting in realistic determinations 
of continuing educational needs. 

Despite the aforementioned problem areas — none of which ought 
be minimized — vo remain convinced that the fortuitous combinations 
of personal and professional acquaintances in our region served tc 
mitigate many of the inherent limits and to negate others which 
might have developed. 
linCOMMLN PAT TONS : 

We feel certain recommendations are warranted, but purposefully 
have not chosen to rank them in order of our personal priorities. 

1, Neither cost nor geographic location sec ns to be a factor in leaven- 
ing physicians r attendance at continuing medical education courses, 
but rather it is the factor of their perception of time away from 
the office and the lack of personnel to cover their practices during 
their absences; consequently, ways need to be developed whereby the 
physician ccn have access to the training necessary in his own area 
for short ei durations of time more convenient tu his way of practice. 
This may well mean a renewed emphasis on providing such educational 
programs within the hospitals themselves. As indicated from the 
findings of this study, nearly 40 per cent of the hospitals in the 
state of Washington make no provision for keeping their physicians 
updated , 

?, A central i zed clearing-house for specialty areas ray provide a 
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nee.h..ir.i sn Cor di ssemi notion of pertinent information to physicians 
throughout the nation. The evidence from this study clearly indi- 
cate - however, that, if journals ere to he a viable neaps of contin- 
uing nodical education, improvements must be incorporated that will 
attract physicians to then as a source for continuing their develop- 
ment in the area of knowledge mid skills, 

3. Mori mphnsis mould he pla< «.*d on and greater attention should be 
given to the promotional aspect of continuing medical education 
programs to develop attractive program topics, and to stress unique 
teaching techniques to be it, corpora ted in a given program. Physi- 
cians appear to be influenced by title topics which indicate 

the relevance of the course content to their particular interests 
and needs. 

4, In a lev instances, the community colleges have been used tn advan- 
tage in developing and administering continuing medical education 
program?; this resource should be developed to a much greater ex- 
tent, It is a flexible institution with great potential as a 
resource for continuing nodical education in the state, 

r ». A more in-depth study of physician educational naeds should be 
pursued in specifically defined areas. It is further suggested 
that .i mechanism be developed whereby those needs can be snbst ant i ated . 

Cn The financial base needs to be broadened in order to add additional 
staff in llio Division of Continuing Medical education to develop 
programs for allied health personnel as well as assistance to tin 
director in the identification of in ids, program deve loprent , and 
P r og ]■ am administration. 
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/. Continuing medical education should, in fact, be a continuous pro- 
cess. The attitude toward this process must be planned for and 
neve J oped as part of the undergraduate training. Continuous edu- 
cation should be built into the lifelong pattern of the study of 
medicine, which night well Include periodical reassessments of 
physician's ability to continue in practice. 

b. From the findings of this study, it is quite apparent that we irust 
use the various media at our disposal to bring educational activi- 
ties to Uie physician, 

?. The design and management of the learning experiences should incor- 
porate basic learning principles relevant to adult education. 

10, There is men for further research to clarify some of the unanswered 
questions about tho management of learning for adults. 

11. When programs are to be evaluated, the evaluation design should be 
built la from the inception of the program and be an integral part 
of the total program, not an afterthought. 

The following points provide brief highlights of the immediate on- 

tie- scene implications of the contract: 

1. Continuing, medical education programs have been evaluated in 
greater depth than at any time prior to contract. 

2 . Problem areas have been brought above the surface and arc being 
wrestled with. 

3. Continuing medical education programs in Washington are charging 
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with emphasis on more locally initiated, individualized programs, 
and use of additional media such as television and radio. 

4. Interest has been stimulated in continuing medical education as 
evidenced by Community Coordinators, locally initiated programs, 
and willingness to assist in feedback efforts, 

5. Entrance of educational psychologists in the area of planning, 
developing, and evaluating health science programs has been 
accepted , 

6. Unparalleled cooperative and coordi native efforts are being made 
between physicians, educators, and medical academicians as well 

as between the University of Washington Medical School, Washington/ 
Alaska Regional Medical Program, fcnd the Washington State Medical 
Association . 

7. The School of Medicine has shown, by substantial financial support 
for the Office of" Research in Medical Education, its commitment to 
leaching and learning. This support adds stability to the Office as 
a long-range resource for continuing medical education in the 
Northwest . 

8< The Project Director, William 0. Robertson, M.D. , and our consul taut, 
John Lein, M.D* are now serving as the liaison between the School 
of Medicine, Washington State Medical Association, the Washington/ 
Alaska Regional Medical Program and nany other groups throughout 
the community. Their relationships among these institutions and 
organizations will enhance fir. me ecu] nvilivo continuing nodical 
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education of fin .3 in this aiw, "Wo talk out and not around 
our pi’ob 1 ons" — and formulate our collective solutions. 

Two points ought be amplified. First, this contract li.is been tin* 
primary stimulus for the development of an Office of Rescan, ii in 
Medical Education to focus on continuing as well as undorg iv.1 . i.e me 
graduate: e ; h 1 • ' , * t i o n . 

Historically, in September of 1967, an Office of Res m 
Medical Education was formally established in the School ui ‘F. .^.u *e 
at the University of Washington. For several years prior to this time, 
the ground work v r as being done primarily through the efforts of 
Dr. V. 0. Robertson, Associate Dean, and a faculty Research in Medi- 
cal Kdueai ifiu Committee. A letter to the Dean specifying the assump- 
tions, objectives, potential activities, and implications of such an 
office clearly indicates the breadth and depth of the scope of this 
concept as early ac 1964 (see Appendix A). 

The two-year period of development of both purpose and function of 
this Office has fortunately received outstanding support from faculty 
and administration alike, which has kept growing pains to a minimum 
Dp to the present tC , a contract with the Public Health Service and 
a grant from the National Fund for Medical Education his provided the 
main financial support for the staff and activities of the Office. The 
supper L fre.;i these tv:o sources during the ensuing critical years is 
now being substantially assumed by the School of Medicine. 

Initially, the main efforts of the staff, comprised of Charles W. 
Dohnei , Pb.D., Director, secretary, and two half-time research assistants, 
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have been _ n pursuit of Public health Service contractual objectives. 
Concurrently, tic faculty has mdo considerable use of the staff as 
they havL ^ l niggled with improving the teaching and learning process. 
These activities have included school -vide and depnr trent a 1 curriculum 
devul opr.iec t and evaluation; conducting faculty seminars in educational 
p r obi oms ; : o n a nit ing with individual f a cu 1 1 y mem!: e r s ; and s e r v i ng on 

a variety f faculty committees. The first Annual Report. of the Office 
activities is in Appendix U. The demands being placed on the Office 
reflect an increasing involvement with the faculty and faculty commit- 
tees in pursuit of answers to questions related to curriculum, teaching, 
and learning. The Office functions as an institutional resource for 
information, consw Itat ion, advice, and assistance to be offered, not 
forced. The objectives for this Office, as recorded the Annual 
Report, encompass the total educational program of the School. 

The efforts of t ho staff arc being expanded In the area of contin- 
uing medical education by providing evaluation services for local Region- 
al Medical Program educational projects and assistance to the Division 
of Continuing MedicaL Kducation within the School and continuing medical 
education endeavors bnroughout the region. Several seminars and work- 
shops arc being planned for the coming year centering on educational 
planning and evaluation for project directors and community coordinators 
of continuing medical education pro jrnTS. Some of these programs will 
be made available to individuals throughout the Pacific Northwest, 

Secondly, as a cons-- quoin e of the contract and ;•••; noted repeatedly 
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during th*.- pursuit of this study — emphasized by such individuals as 
Sir Theod re I* ox, Van levei'i: Busch, and Peter Woodford — the* actual vole 
of medical journals in continuing medical education differs to a con- 
si derahlc fMUnt fron their potential one. iSut even the actual role 
proves entire! and basic for most physicians' continuing medical 
education .*:'fctts. The challenge then exists to develop modifications 
il medical p urna is*— format , style, or content--to maximize their poten- 
tials or possibly to 11 imprint" nodical students tc react differently 
to them. To that end, ve are attempting to undcrtr.he a rather detailed 
study oi consumn -phys ic i an reactions as veil as' professional opinions — 
drawn fro;:, the field of journalism — regarding the existent national 
and speciality medical journals. We would propose to develop programs 
to build on their strengths and minimize their deficiencies. Such 
would be pursued on the assumption that the resultant changes would 
lead to favorable reception by consumer-physicians and more effective 
opportunities for learning on the part of physici an-suscr ibers through- 
out the region, It would also be our premise that the generic model 
developed under this approach could be translated into action in other 
regions of Lire countvs vote such considered desirable. We feel this 
avenue of investigat i on is long overdue. 
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September 30 # 19,>V 



Dr. Jo -m tt. DognesQ# 'Jean William 0, Robertson# M.D, 

School of Modi cj-ie Medical Director 

Proposal re "Research in Medical Education 1 ' 



Ac you know, over the past 10 yearn medical cchociFj have bee»v oxen in in.' 
their educational fiinctiono - documenting# appraising# and evaluating thou. 
Sparked by the A.A.M.C, and Oeorgo Miller's group (Buffalo mid Illinois)# 
cuch programs are adding a dynamic and objective aspect to "teaching and 
learning" which permits this’ function of Medical Schoolo to achieve a 
caup; tativo status with the conventional "recearch and pet-’ent care" function 
to the benefit of all concerned. 

At present# several R?par taints In the School of Medic In a have already 
undertaken analyses of thoir educational prograr.6, An additional emphuoio - 
on a school -vide basis would appear indicated, Toward that end - i,e, the 
irplcnwntation. of a more fonnalised program of "Research in Medical Education 11 
1 would propose: 

1. A number of assumptions be examined in teiTi3 of the likelihood 
of their being accepted - or at least not actively rejected - 
by the faculty and administration of the School of Medicine 
prior to the Institution of any program. These includes 

a# That tho complexities of the educational process are legion. 

b. That these cooplexitieo are au appropriate area for 
scientific investigation} proper investigation 
provide either (or both) theoretical knowledge about# 
or practical approaches# to solving problem f contained 
theioin. 

c. That the educational process in medic ino - undergra .j/Xc, 
postgrad) vat e# and continuing - lends itself in on rrxtraordinary 
fashion to etwdyj appropriate study of it would affect# in a 
constructive manner# the medical school’s success in its 
objective of providing optimal learning situations and 
experiences to all its students. (Moreover, no with patient 
care# the incorporation of a research ^rogron aeons vo Ij, [ prove 
overall function.) 

d. That# considering tho total budget of tho Medical School# a cm 13 
pcrcentogo devoted to '’Research and ]>avelO]nont M of ito product - 
successful learning - would seem appropriate. 

As I ocq it, none of our faculty would oppose cuch na&uinptions and# 1 
believe# all would favor developing a noro fonral program. 



Dr, John R 
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• Helena, Dean 
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2. Accepting thccc necir- ptlor.t c.n CCfico or Division of research 
In radical Education rrlcht Vo forma!?.:/ cr/cnblichcd as a "ctaff * 
(c.3 opposed to r\ '‘lire” ) activity, Corccidvnlly I'd coo It cxo 
a Dean 1 a Office related function tot vitb participation by 
irllTldv.nl department?! ::c;\bor.: . Cveh vovld evllau for Dchool- 
vido o tv. '"lea go veil to erharciivp the effcctivcnoao of intru- 
de par c rental act ivitics * 



3. TV o objective a of cv.ch a program vculd Include: 



o, To serve no the stirrolve for research in nodical education. 

b. To provide a "ccrviec", or* access to appropriate cervices 
or consultative aervicon, for Individual faculty reenters 
or to ir/JlTiduvl departments vJio vioh to conduct intru- 
de rjamomcntal invest lections* 



Ct To coordinate or assist in coordiratin .3 or to conduct cchool- 
vidc or interdepartmental In ve s t 5. cat ices • 

cL To assist in cecwir:^ nocooeery ihinic for support of 
investigation's. 



c. To provide cr assemble appropriate data for committees eve a 
as the Cuiriculue Commit oee - vh ich, In turn, wuld rrhe 
than available as an objective basic for their deliberation. 

f. To provide a fo^al point for nonccdical school Uni varsity 
faculty n embers to turn should they to interacted iv the 
cubjcct of nodical education. At tfco came tiro ouch o 
pro^rc-a would provide ux avenue of rapport for nodical 
Gchool faculty nenbcrc vith there in education, psychology, 
cocio.loay, cccncaica, engineering, loathe: in ties, etc* and 
allow mutual assistance in the pi anziirp and c >nductin£ of 
any cooperative invostiention involviTj both groups . 

G. To verb vith ancillary services - such us Audio Vi cull 
Educational Services - in fiuidierinp the investigation of 
thoir success in achieving their objectives* 



b» To or ole t in th r azcoc 
in the curriculttn* 



:nt of nev progra Mj or innovations 

i. 



To assist the School Df wcdicim tr.s it aropco with the 
immense problems of both cxadvnvbo^ coat irtdsG education. 



To further the development cf optical learning citvaticn in 
the nodical school sotting* 
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IY. John ft. Eo ; 



Dean 
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Scpberibcr 30 , 19#+ 



h • To irrelcnervt such a pro:;ra~i the 9 olla\:ir>£ otopo vould ccc.ii 
In order; 



a. Designation of a rccp'ccn. blc indiviOiuil cn a temporary or 
pennnent basic. 

b. Ectabl J ohnent of lie- on vdth non-Ifcdlcal School University 
Pfccvifcy. This hoc proven of cnorroua value to successful 
profYaoJ c lee where. 

c. .Y-iv^uierricnt of the proeprra to rrodicel school faculty and 
student body. 

d* Appointment of an Advisory Covrdtteo riT-renentirvs tho School 
of Medicine faculty - virion vc?;Utd ccct '•ith an open door 
policy. 

c. Catalo^las of ctudics of nodical cca-.co.tior. already in 
pxo^rcoQ. 

f < Inventorying of individual faculty ror.be re and depr.rtrcntal 
latere eta. 

G* Duvelopir# - in the li^ht of cuch factors as faculty Interest, 

Goo Graphic location, physical facilities, ctcr - tho crdrter.ee 
of potential avenues of investigation at different educational 
levels vhich vould appear coot prcaiclnj. These levels vould 
include: 

1* Pro nodical education. 

2. Undercracheate ixn . ui cal education 
3* Roe car eh training pi^cromo 
4, Internship programs. 

5* Ho si coney programs • 

6. Follovsbip cn*l post -do choral pro£rcna, 

7. CcntimiirG r;odj cal education proGYcrn# 

8. para -nodical procrevaa# 

h. /vanG areas libel y to be fruitful an initial studies the follo.'ir.G 
vruld seen ripe for iuvostifption: 

1. An cicsoscucnt of the ai^dfiennee of cocial- cultural bacltt-^r^d 
factors cs those appear to iriflvcnco undergraduate ctv.d.eat 
attitudes, performance, and pools of the University of V’asMrjton. 
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2* An appraisal of tec Jnportnr.cc of rsccUficblc detractors - 
1T noico M - in interfcrirs vita optimal learning cn tha pert 
cf undercpmluato end poetpraalvmla ctv.dc ate. 

3, /a gssc: mart of tie irrrc.ct of nodical journal c cercsni c a t i one 
on the ?5 reet pir.f ;r.te Xcrmlr a ? cf practicing physic:*. ran. 

(respite c. tror^ecrlrc lh,0f0 r.felirheu ar.nucf.iy, us of !Sc2 
only 82 studies have been rr.ih.t in this area - u^th 79 directed 
solely r.t the inp~ct of edvaetisisj there in I } 

4 . The construction cf a theoretical mi notifiable no del 
re:<rce':;i.‘2\:* the ; ymaaics of o sveco-* sful "rounds"* 

9. Time cud notion studies of appropriate lcamir.5 situations. 
Additional points could Ij listed; their listing, I feel, 
should or.. -ait l reefer f acuity involvT.r.o:rt# (As is evident 
ot lD.in.oie, the faculty itself mat be heavily Involved 
in cuc}3 p.Vminp cr the protean my fail for lech of 
interest - or prorcnca of threat.) 

6. Aa araLycic of written and verbal ccmruni cat ions in 
ncdicino. 

7 * An appraisal of rro^rcrr.ed lonralrc, T.V., etc., ns eh esc 
techniques develop. 

5, Ca c more practical level, the follcvins schedule nlcht be appropriate: 

o. Further diccuscicn vlth non*Kc 4 ical school faculty at the 
University of their potential interest in cv.cn a pro^oren and 
their villi n^nens to pro 'vide consultative advice as veil re 
their joe: i bio pext ic ipc.t ion « This copect cannot be errphnsined 
too strongly; rare the College cf Education to be interested in 
boccriirr involved, I feel the School of Kedicino react bo 
prepared to provide appropriate support# ir<r/orthelcc3, ouch 
involucre ; it meet c.t uo tire nir Arise the primary need of active 
participation by Cchool of Medicine Faculty; to bo dynamic in its 
effect cuch is cruel cl. 

b. The f o naul isc. tie n of the pro^ren co that q preliminary announverent 
niebt be ! rule to the Faculty - cues niefrt be concurrent vith Jr- ch 
Licr^c arrival. 

c. Too apjoiiirmt of a moll (rather then a lares) "Advinory Ccanittcs" 
vitli r rebel's rerr.h^ at least initially, responsibility fer 
providing fee die.'; to departments other than their cun. Fcvbopa 
five or si;-; r.r n- Ccjc.r c~ eatol ch adman rarold suffice; such vvvJU? best 
be able to develop a creirer format for 3-6 nenths as the pre;™; 
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Dr, John. P.. Eo jr.r.-T, Doan 



- 5 - 



Septeriber 



a. c-r.tnlo-ins o: ctv.oicn currently in progress to be 

undortchea rbortiy after foior.tliroticn. 

c. Tag ornroaefeiny. a? rcnroccrtativa foi'r-.".o;loo.o for "iYaCArc" 
of "invontirritiorsr, (vovl<i also to vr.fortcJ;cn chortly after 



f orr,un nnticn. 



6* 7o f.ot the off the c^^-'-rdj hr.vo .space j I* ex cciticip^vO 

Tixf.xixly rocviire cr/ce £ r h?-l t ;.CT. c \ ccereicric.X. cervices itoln 
three" rcifcM or initiation. 

Xn vr; r. v : T f d ocr r. .v.ove active rresren In ?f*~ccrch in Kcdicel EdrccAion 



->q+, rlr^y cv' *? ^ to hcovl c cV* e Vuv ccv/iliy 3c. pee vVj , ^iy c-orvin^ n n c..^. 
;t/Jirib?«ticn of the teaching function "of the Echoo.l of i'cdiclre. X wuld hop 

Sincerely, 



ct 

you concur* 



VftXXlco 0. So^ertsoa, M.D# 
I to di col 3Qirccto? 



VO^SCDO 



^ xV,v-Y.<- 

j/ o ynj£C*v 
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COMP U T E R- A I D ED INSIKUCHO:; i;-; li/ART , CANCER, 

STROKE , AND KLLATi. ) UhiABBS 

This project itself was carried on under the aegis of the Wash- 
ington/ Alaska Regional Medical Program. it;; purpose was to develop and 
then evaluate computer-aided Instruction (t U) as a complementary and 
supplementary method of continuing education for physicians, nurses, 
technicians , and other health pjrson.nl, j ue project*, wjs directed by 
Mr . Don .Tiller, Project Coordinator, wa, hvoloped and supervised the 
production and the implementation of nine CAI problems during the first 
year, These programs ares "Arterial Fibrillation,* 1 "Arterial Flutter, 1 ' 
"Bradycardia/ 1 "Introduction to BCG's," "Malignant Melanoma/' "Renal 
Function In Pregnancy/’ "Renal Hypertension," "Review of Arrhy tfiuiins , r 
and "Tachycardia . ’* 

Our office staff, supported by this Contract, assisted Mr, Miller 
in the project evaluation. The following is a synopsis of the field 
evaluation as prepared by Mr, Miller, The communities of Aberdeen, 

Tort Angeles, Port Townsend and Anacortes, VJashir.gton , were selected on 
the basis of their wil lingness to cooperate and thclu remoteness fre.n 
Seattle. Duration of availabi lit)' of equipment was determined by the 
number of physicians in the community arid corputer time available. 

Users were informed that they could use the system on weekends by spe- 
cial arrangement, or late in the evenings, if they let tile computer 
operator know what time, before the equipment was left in each commu- 
nity, it was demonstrated and a "hands o:i" experience wis v-ade available. 
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Instruction sheets were left along with the phone numbers of where help 
was available if trouble developed. 

Evaluation information was collected as soon as possible after the 
equipment was raoved from a given ci^y. In Aberdeen and Port Townsend, 
evalua:ion meetings were added to the agenda of prc-sched‘led conferences. 
These communi t i es provided the most feedback because most of the parti- 
cipating physicians were there. Dr. Charles W. Dolmer from the Office 
of Research in Medical Education and Mr. Miller attended all evaluation 
sessions and coHected verbal comments at the meetings and also respon- 
ses to a ques t ionnair e. 

Iti general, satisfaction was expressed in use of the system when it 
was functioning properly. The greatest hindrance to the machine's use 
was seen an being the time of day that it was available. Data supplied 
by Mr* Miller indicated that the more times a day a system can be used, 
the more it is used. 

Ev aluation Fie Id Trials 

Attached is a form with the categorical items summarized (Table i). 
The open-ended responses are summarize! below, 

Question 1: What opera tio nal difficulties , if any, J Id you exper lence? 

In general, all who reported use of the system expressed their 
enjoyment in its tse, as long as it was functioning properly. However, 
failures were most frustrating. It was very difficult to contact the 
computer room, especially at night. Failure seemed to be caused by a 
variety of sources-- terminal operation iguoranc. , computer-operator 
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error (i.e., pulling disc in middle of ran), phone company ignorance, 
instructional program errors, computer failure, slide projector jams, 
and electro-mechanical failure of the terminal. In short, just about 
anything that could havr* happened did happen, at least once. 

It was expressed that , after a long day, the physi:ian or nurse is 
tired and that light material or humor is essential to keep up the human 
Machine dialogue. The greatest hindrance to the use of CM was the time 
of day it was available. The more times a day the system can be used, 
the more it is used. To support this, figures from Aberdeen and Ana- 
Cortes are presented (Table 2 attached). 

Also the users found it difficult to be at the machine within the 
15-nlnute window that was available to them. If they did^t start 
within 15 minutes of their scheduled time, the computer operator assumed 
they were not going to make it and started a ‘'batch 11 process. Other 
problems were identified, such as a need for better labelling of slides, 
better identification of trays, and some of the loops designed for 
review did not answer questions needed to be answered. It was expressed 
tli at the CAI was espec ially effective in causing t_hc l earner to be 
a ctively invol ved in the learn lug pr ocess . The material level 9eemr.d 
appropriate for most of the programs. They also indicated that a wider 
variety of programs would have been advantageous. This, of course, is 
a little difficult at this tlno when only nine programs have been developed. 

Some of the other advantages mentioned were: they did not have to 

expose their ignorance and could still learn; the typed -out answers to 
take home w T ere good; a variety of difficulty of questions was presented; 
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the pressure of one-hour limitation gave them a sense of urgency. In 
general, satisfaction was expressed in the use of the system when it 
was functioning properly. 

Question 2: What othe r prog ram, i_f any, wou ld you like t o have on CAT? 

The response to this question seemed to bo related to personal 
Interest and no trend could be established , 

No. of Requ ests Program 



Other requests were for a more diversified use such as a diagnos- 
tic aid; handling a , uysician’s medical records (and medical history 
acquisition) and paLicnt instruction. 

Part ici pan t s 1 Sug ges t ions for _Fu ture Field Opera tjLons : 

1. That there be another phone line to the computer room. 

2. That all odd computer times be made available (i.e,, between 12:00 
and 1:00 p.m., 5:00 and 7;0Q p.m., and nay as veil go on until 
9:00 p.m. since the regular start was 7:30 p.m. After midnight, 
it nay as veil be left on for use in the early morning hoirs by 
nurses, etc. who arc up then). 

That data sets be used at both ends to cut phone costs and ease 
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Drugs Used in CCU 



Fluid balance 
Childhood Diseases 
Skin Conditions 

Respiratory Treatment (use of 0^ te.n) 
Shock and Patient Treatment 
Endocrinology 
Acid base Balance 



3. 
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difficulties in getting coupled dp- Perhaps a plastic dial card 
would facilitate this operation! This was one of tiie most consis- 
tent troubles. 

4. Leave a stand-by terminal in the field as a back-up. 

5. Remount all slides in plastic (now under vay) . 

6. Have demonstration and evaluations as part of regularly schedulcu 
necti. . 

7. Since the feedback to the system is the primary route to improve- 
ment, the validity and reliability of returning information is 
critical- The quality of this information can be enhanced greatly 
by additional programming- Three cases of information nee: 1 this 
attention . 

a. lining: when the program was taken and how long it took. 

b. All the information on the present questionnaire. 

c. Educational strategy sophistication. 

d. Need error count form- 

e. Need error count individual error pattern through program. 

f. Need latency form for end of print to response. 

g. Spaced review. 

h. Ability to print error ratio on individual request. 

1. Program to detect legality of CA1 addresses. 

j. Program to prevent the election of an invalid frame number 
from killing the system. 

The second part of our office's involvement with this project was 
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the evaluation of the use of CAI with undergraduate medical students 
and nurses. Tile Office helped to develop and analyze equivalent foiiiis 
of examinations to be used in pro- and post-testing utilizing an anal- 
ysis of variance design to measure the amount of knowledge gained as 
well as to compere the CAI quantity of knowledge with programmed instruc- 
tion and traditional classroom lecture method. Four of the CAT programs 
were selected co be used in the study of comparison gain in knowledge 
with second-, third-, and fourth-year medical students and nurses. 

These four programs are as follows: 1) Malignant Melanoma, 2) Renal 

Function in Pregnancy, 3) Renal Hypertension, and 4) Introduction to 
ECG 1 s . 

Problems related to the administrative aspect of these parti- 

cular programs with the result that analyzable information was available 
only from the "Renal Function in Pregnancy" program. However, a*, addi- 
tional study using a "Malignant Melanoma" program was carried out later 
in the spring under the direction of Or. John M. Short, Assistant Pro- 
fessor, Department of Medicine. This office also participated in eval- 
uation of that program. 

S tudent Evaluation : 

Medical students uer; used to collect information on the relative 
merits of lecture, programmed instruction, and CAI because it was hypo- 
thesized that students would afford a more controlled situation than 
practicing health care professionals. These arc the same students vho 
provided a testing ground to improve* the materials. 
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The first group of students was taken from the conjoint course, 4 5 ^ * 
Thirty-four examinations and comment sheets were returned from the lec- 
ture group, 12 from the group that had computer-assisted instruction 
and programmed instruction, and 10 from the group that had programmed 
instruction only. The reason for overlap between FI and CAT for the one 
group was tbit »>ach student was limited to on*-half hour on the computer 
terminal. Table 3 attached is a summary of the returned questionnaire. 

There were several notes on the questionnaire showing the desire 
for more personalized learning situations such as a combination of the 
media involved. The results of the questionnaire indicate that the stu- 
dents in the lecture method were more satisfied with the material and 
methods than the students on CAI and PI for this particular experience. 
Nevertheless, all groups wanted more CAI and PI but fever and better 
lectures. Because of a limited number of students involved in CAI and 
PI and just PI categories, test significance for mean error levels would 
be meaningless; nevertheless, on the face of it, there did not seem to 
be much difference in mean erroi levels. Another problem was that the 
test had a very low "ceiling" and thus several persons got the entire 
examination correct. 

The results of the two courses, "Renal Function in Pregnancy" and 
"Malignant Melanoma," that were anlyzed by this Office arc summated on 
Tables 4 and 5, attached. The reliability for the equivalent forms 
tests used in the "Renal Function In Pregnancy" study was .60. For 
Dr. Short's program, the post-test consisted of the pretest plus 23 







additional Items. Reliability of those two instruments was .17. As 
shown in the tables, there wore no significant main effects and the 
Y ratios were non-significant. Thus, the null hypothesis was not rej 2 c 
ted for either study. In addition, there were no significant differen- 
ces for tine factors in the Regional Medical Program study. These 
results concur with available studies in educational research (Stern, 
1963; Mchcachio, 1%3). 

For tie program, "Malignant Melanoma," the second-year medical stu- 
dents were assigned randomly to three sections. The same program was 
presented in three different nodes of instruction — CAI , PI, and "tradi- 
tional lecture." The final number of students in eacli section was 25, 
24, and 31 in CAI, PI, and lecture , respectively, due to program change 
Information was collected reg rding t lie student’s attitude toward these 
three instructional media. 1) The students were asked to indicate an 
overall qual ity of the program by rating it cn a scale from "poor" to 
"excellent"; 2) the preference for various modes of instruction was 
determined by asking them to rank the media used in order of their pre- 
ference ns a means of instruction ft om lowest to highest; and 3) the 
attentio n vwilue of these tnree modes wa 5 determined by asking the stu- 
dents to rate them on a seal 2 from "no attention" to "excellent atten- 
tion value." 

In a previous unpublished study by Dohner and Hamberg, "Selected 
Characteristics of MedicaL School Students at the University of Wash- 
ington," programmed instruction was rated as most preferred wlion stu- 
dents were asked to compare various media that they had previously used 

lav 
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prior to entering medical school. Programmed instruction, filmed 
ins true t ion, taped instruction, television instruction, and computer- 
aided instruction were rated in that order. The percentage of students 
who had had experience with these various media are as follows: filmed 

instruction (99 per cent), taped instruction (85 per cent), programmed 
instruction (58 per cent), television instruction (53 per cent), and 
computer-aided instruction (17 per cent). It is not clear if these 
ratings are the result of novelty of the various media or of satisfac- 
tion or some other variable. It is interesting, however, that program- 
med instruction, while third in familiarity, ranked first with regard 
to preference for those using it and that television instruction ranked 
next to last. 

Based upon these findings, it could be hypothesized that programmed 
instruction vouLd receive higher ratings in terms of preference than 
either CAI or lecture method* It would be of interest also to see if 
PI were rated highest using other concept variables. Subsequently , 
the following hypotheses relating to these three nodes of instruction 
were set forth. 



IfYPOTllESIS 



1 : 



There is no difference in the overall qua lity 
rating by students between the CAI. PI, and 
lectrre modes of instruction that they respec- 
tively experienced. 

There is no difference in studcnts t preference 
ratings between the modes regardless of which 
mode was experienced. 




lui 



HYPOTHESIS 2: 
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HYPOTHESIS 3: There is no difference in student preference 

ratings between the respective modes experienced 
by the students, 

HYPOTHESIS 4: There is no difference in the attention value 

ratings between the respective modes experienced 
by the students. 

The students were asked 1) to give an overall quality ratine of the 
program, 2) to rank the media in order of their preferences as means 
of instruction, and 3) to indicate the extent to which the program kept 
their attention. Ml of these ratings were based on a rating scale 
from 0 to 4, with 0 being the lowest and 4 the highest rating, Kean 
ratings were determined for each of the instructional groups and compar- 
isons were made between the means of the various instructional modes. 

The statistical significance and mean differences on these responses 
between CAI and PI, CAI and lecture, PI and lecture in each of the groups 
was evaluated by means of t ratios, 

Table 6 presents an analysis of the roan differences between the 
groups on the attitudinal responses. The lecture method received the 
highest overall quality rating of 3,00, The first null hypothesis is 
rejected (p^.05) when comparing PI or CAI with lecture* The lecture 
method is found to have a significantly I ighcr quality rating than 
cither. There was no significant difference in the overall quality 
ratings between CAI and PI, 

In order u test the second null hypothesis , the roans of the 
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preference ratings for the three modes as instructional media were 
computed across all students. A significant difference (p^«05) in 
preference ratings was found between CAI and lecture method with the 
lecture method receiving the highest preference rating. There was no 
difference in preference found between CAI and PI or between PI and 
lecture . 

The data relating to the third null hypothesis are found in Table 7. 
With the exception of two comparisons made by those in the PI section, 
no differences were found in students' ratings of the three modes as a 
means of instructional media. The results show that the students in 
the PI section rated the lecture method higher than CAI, and PI higher 
than CAI. This type of discrimination did not appear in the CAI or 
lecture group* 

The fourth null hypothesis set forth is not rejected* Hone of t lie 
mean differences in students 1 ratings of the attention value of these 
programs is statistically significant. Based on these data, there is 
no evidence to suggest that a difference in attention value among 
these three modes of instruction exist, at least for the population in 
this study. 

Comments f r om those s tu dents In C AI mode : 

The CAI would be better at holding attention if there were sound avail- 
able also. 

Each question tends to be a discreet entity and a good deal of proactive 
and retroactive inhibition is in effect. Noise. 
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It could be that cramming for finals all day has made my attention span 
very limited, but whatever it was, I was unable to really concentrate 
on the program. (The tine of day mi t ht be very important,) Having had 
no prior experience with a program of this nature, I found the computer 
completely overwhelming and hard to concentrate on. I would be inter- 
ested to see if a second exposure, would be more profitable, 

I like the ability to be an active participant in my learning, 

I will be better able to evaluate it after several weeks or months when 
I see how much I forget! 

One trouble with this type of learning is that the student is left with 
no concise material for study at a later date as one would have from 
class notes, A tremendous improvement would be an outline of the 
material to be received after the CA1. The time 1 really learn material 
is not on first presentation, but on later review. 

The ending was rather unclear — would be better to be informed that the 
end has come. 

The program is gcod--as a variation from usual methods of learning. 

The variation of methodology Is effective in keeping one’s attention. 

Fun! But a second try at it would be helpful—bu t might break your 
pocke tbook. 

Typewriter was distracting, Questions seamed lacking? One isn’t able 
to look back to refresh while taking CAI — as one could with PI. 

Interesting but seems to jump around — no smooth transition in points. 

No . 

I think that a person certainly must be in the right frame of mind when 
he takes this program. Otherwise, it could be very confusing. Also, 
the knowledge gained from such a program requires rather intense concen- 
tration if any of it is to be remembered. This is difficult due to 
several problems Including 1) noise of output typewriter, 2) inability 
to go back to previously done questions for personal (rather than pro- 
grammed) review. Of course, this type of learning does not permit 
questions or any in-depth probing of any subject introduced. (This c<in 
be done in lectures, texts, seminars.) The slides presented were dif- 
ficult to view due co lighting, angle of projector and general inconveni- 
ence of changing slides. 

I found the program easy to follov.*, but ft-und I could rot jecall much 
of the detail required by the test. 
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Yes, finally you asked . I got to the pink sheets and t nought, "Ah hah! 
Now I am going to get to say something. But no--all I get are "circle 
the number, check the box, mark your preference.’ 1 

The computer I was uncomfortable with. This nay go away with use. The 
typewriter is very hard to concentrate around. On the whole, the pro- 
gram *was not very interestingly written. By far most significant for 
me, however , was the lack of opportunity to feedback more than one 
flick of a finger. Believe it or not, I do think during lectures and 
while I am reading. And I get a chance to write something that comes 
from ny regurgitation circuit. Here I haven’t felt rewarded signifi- 
cantly. The test was comprehensive but I kept thinking, "This is an 
awful big favor I am doing," because it just dragged along. 

Several o r the questions were vague and/or presupposed a prior knowledge 
that was not given in the program. The inevitable delay of typing also 
caused a loss of attention that is undesirable. Also, some of the ques- 
tions are entirely too simple. 

As an overall effort, however, yon are to be congratulated for at least 
trying some tiling new. 

Good potential in some areas. For topics that involve clinical obser- 
vations and differentials that can be confusing, I question the use of 
it. I find also myself doing the m^chinics of the program but retain- 
ing actively very little. I think a seminar with slides both histo. 
and clinically in small grooms is the only way in dermatology. 

Although the technique is interesting, once I left the computer I felt 
I had l’.ttle knowledge of the subject. 

My feeling about learning is t ha t to learn best and fastest, and to 
retain that knowledge , a person needs to use all his available senses 
(bearing, seeing, feeling, smelling). The impression in this way has 
to be greater. The computer program seems to eliminate most of these 
impressions and makes the experience mechanical and transient. 



Common t s from t hose st orien t s In PI mode : 

I thought one of the objectives of a programmed text was to allow each 
Individual to proceed at his own rate; whereas I sat so long waiting 
for the slides to change that 1 forgot the material I had read. By the 
time 1 finally got to ti.e post-test, it was basically just another 
guessing game. 

My case I;i perhaps teaching new’ nath to an old nath mind: T have a 

good deal of trouble consolidating the programmed material. 



It did lion seem to flow naturally. Many times one just fell into the 
next frame — a logical development was not felt to be present throughout. 

The programmed text was not completely explained at first (how to deter- 
mine which box to go ic next?}, hut after it was explained it was easy 
to follow, I probably would have gotten even more out of the program by 
taking a little longer to absorb the material in each ease, Once a stu- 
dent is more used to this type of instruction, his time will be norc pro- 
ductive and he can proceed at his own rate. 

Once over the material in insufficient, 

You should In more careful to give instructions on how to answer the 
questions, i„c., how to read tli* answers. Also it night help to point 
out a few rc-pro sertative tilings on the correct slides — after we have 
chosen one cr the other--it Is possible to gu ^ss the correct slide of 
two not so clear slides and not know why it is right. 

It would he considerably improved with adequate projection and with a 
typed summary of signs, symptoms, treatment and prognosis of each 
disease placed at the end of instruction. 

The instruction should have been reworked, 

Programmed instruction needs some drastic . roroveraents. 

I felt rushed. 

Vairly v*ell-organized and instructive. 

I find PI done i .* my own pace an extremely efficient way tc learn, This 
program was not bei.eficial foi we becauj e of the forced rate to keep up 
with the slides. 

Not enough tine. Can't WDrk at your own pace because ot the slides. 

Would be great for s r If -teaching. 

1 do not feel relaxed or as if I enjoyed this last two hours. I do 
feel seminar accompanied by slides and with syllabus would be mere 
a Jvantageous, I would prefer to he given the program text to do on ny 
own at ny convenience. 

In devising f he program I feel that mere sequential ordering of ques- 
tions docs not provide enough recall from the student. In devising 
this program I would hove put in come comprehensive questions at the 
end of each patholo/ical entity. 



Subject 

Sequence qaes 1 ion 
Comprehensive question 



I 




II 
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I had trouble evaluating the program per se because 1 dislike program 
texts so much . I would nucli prefer to have the same material presented 
in 1/5 the space, memorize some definitions and characteristics, see 
some pictures, and leave the subject for a while before returning to it 
to relearn it, My retention following the use of a program text is mini- 
mal plus 1 hav^ not had adequate chance to compare various aspects of 
the subject so that 1 do not develop a good perspective on the subject. 

I would spend iay tine writing a clear, concise, veil-organized syllabus 
(which the Dermatology staff has already done) rather than a programmed 
text. 

It would he more effective for mo if there were more review questions 
throughout the program. Otherwise, one night as well read it in a book. 

At first, the directions of how to get from one frame to the next in 
the programmed text were unclear. I prefer to read the material anJ 
take notes on It at ny own speed in a text to a programmed text. I 
don't like learning under pressure of any sort. 

I don ! t feel I hod tine to really learn it, because of the slides, you 
had to stay with the group and coulrrt think things over at your own 
speed . 

1) Original instructions on hov to proceed were obviously unclear. 
Hovcvei , once explained, the program was easy to follow. 

2) Each individual needs his own slides to facilitate immediate review- 
ing, comparison, and to allow this to be an independent learning 
experience that each student can adjust to his particular pace and 
mode of learning. 

3) In statements that list criteria in sentence form, I find it diffi- 
cult to see and remember these criteria. My feeling is that it 
would be ' quicker method of visualizing and learning criteria if 
they were listed in outline form, c.g. 

K.ily lentigo maligna... 

1) has atypical cells 

2) appear only at dermo-epidermal junction 

3) do not invade dermis 

4) Inflammatory infiltrate sometimes 

Keep up the effort for improved learning tcchniquos--very very 
tired of poor lectures. 

It was a good program. After reading through it there should have been 
discussion (seminar) before taking exam, 

As an experiment, tills course has been interesting; certainly it breaks 
the monotony of a lecture schedule. If the programmed instruction is 
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continued to be used, I would suggest an improvement in the logical flow, 
I have the feeling that some lesions have floated past without a defi- 
nite concept of them, I would like to have been able to have an instruc- 
tor clarify some of the descriptions and distinctions for me, 

I found that the c/an asked for fine distinctions which were not apparent 
to me in the program. 



Comments from th ose students in Lecture tnod_e : 

Some of the questions on Line test were not mentioned in the lecture. 

The enthusiasm of the instructors stimulated my Interest (and attention) 
in and to t lie material covered thus far, 

ihe lecture seemed too much like a computer course, It presented ques- 
tions that the ordinary lecture doesn 1 t--which is the main problem of 
the ordinary lecture, e,g., you would show a slide of Junctional nevus 
several times to reinforce it, whereas ordinarily the elides would be 
all in one slug and not integrated among others to contrast. 

Dr, SagebielVs presentations are lucid, concise and a pleasure to 
attend, 

I thought I knew the material from the lecture before the test --but the 
lecture was cither no* complete or the tes v covered superfluous material. 

It was a headache to answer all the questions, otherwise OK, 

From the practical point of view, programmed instruction (paper and pen- 
cil) with carrousels to show visual examples seems to be the best 
approach . 

In essence, I feel lectures are a waste of time. The reason the aver- 
age student likes this is that he is told what is important and it 
takes less work than reading vast textbooks trying to sort out what is 
relevant. If he were first directed to pertinent material and Mien 
shown clinical or slide material to reinforce major concepts in small 
seminars it would be an improvement of this lousy spitb3ck rote memory 
system. 

The efforts of the Dermatology staff arc apprcciateu. 

I like the way the lectures on the first day were broken up. I think 
that it if much better to have 15-20 minutes from one lecturer and then 
change. This offers two rain advantages; 

1) Lecturer doesn't have tine to ramble; lecture must be short, 
organized and concise. 
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2) Student doesn't have time to go to sleep. 

So far program is great. 

The "programmed instruction" was written confusingly--! t takes longer 
to figure out direccions than learn the material. The slides were dif- 
ficult or inpossible to see. My past experience Indicates that this 
is the si owe- st means of learning 1 have ever attempted and in this case 
ny anger at che a^ovc circumstances made ii impossible, 

A good lecture is a tremendous learning experience* Most lectures, how- 
ever, are givin, i» seems, by someone who v;ould rather be doing something 
else, 

The sheer number ot lectures is a disadvantage. It's hard to consis- 
tently maintain interest in any single teaching method* A variety is 
need jd . 



F utu re P l annlng : 

There seems to be a sufficient amount of enthusiasm among physi- 
cians and students to want continued work in this area. Future eval- 
uation of CAI programs should involve closer work with the evaluation 
area in the development of the programs themselves in terras of objec- 
tives with the development of the lnsttuments in order to build instru- 
ments that possess higher reliability for evaluational purposes and in 
the actual implementation of the design after the selection. Coopera- 
tion of all parties involved must be secured long in advance of the 
implementation of the program* 

discussion : 

The results of our evaluation lead us to suggest that there is 
probably going to be no real difference in the amount of knowledge 
gained when different methods are used and that any valuable differences 
will be dependent on other factors, such as time involved, convenience, 
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efficiency, and cost. However, there still remain many variables that 
certainly were not controlled for or examined by this study, such as the 
effect of different levels of I.Q. and times of day, levels of branching, 
sophist ication, etc. If one simply views the CAI method in terms of its 

ability to teach an amount, i.e., quantity of information, results of 
our evaluation confirm that it functions in this manner as well as pro- 
grammed instruction per s£ and the "traditional 11 lecture method. 

The fact that no difference was found between CAI and PI in either 
quality or preference ratings is in contrast co the study of first-year 
medical students at the University of Washington (1968) when PI received 
the highest preference rating among various media, including CAI. It 
could be hypothesized that the two groups are from different populations, 
or that the quality of the PI and CAI programs contributed significantly 
to the apparent shift. We did not test this hypothesis. It has been 
quite popular recently to deplore the lecture method as an outdated and 
ineffective means of disseminating knowledge, even to the extent of sug- 
gesting it be completely eliminated In some cases. However, the analysis 
of the data in this study tends to support the lecture method in both 
quality and preference, lz -.east for the topic, "Malignant Melanoma." 

This is a reversal of the expectation that programmed instruction mate- 
rial would have been preferred. But, again, this phenomenon nay be d-e 
to experience and familiarity with the method. It is also of interest 
to find that students did not feel any differences in the attention 
value among the media. Apparently, even the novelty of the new approa- 
ches to teaching had little Influence on student attitudes toward them, 
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although we did not test this hypothesis, £pr 

The only significant differences found in student comparisons of 
the three modes was in Lhe PI group where CAI was found to receive sig- 
nificantly lower ratings than PI or lecture. No other clear distinction 
was found which suggests there may not be any clear pattern of prefer- 
ence among the instructional media used in this study. Other studies 
(McKenchio , 1960; Koenig et al, 1959) find that methodology oes net 
affect learning. 

Based on these findings, there is little evidence to suggest that 
students 'will learn more effectively or that they are likely to be more 
highly motivated with any one medium. The quality of the program or 
the topic to be presented may, in the final analysis, be more important 
in deciding which medium to use as well as the medium itself. The 
instructional media have no inherent magic in and of themselves to 
enhance the learning environment. 
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PHYSICIAN RESPONSE PROM TWO CITIES 



COMMUNITY 


PHYSICIANS 
IN COMMUNITY 


DAYS 

IN COMMUNITY 


HOURS USED 


NUMBER OF TIM S 
AVAILABLE PER DAY 


Aberdeen 


22 


25 


20 


7:30-8:30 


Anacor tes 


7 


6 


50 


7:30-8:30 
plus time 
requested from 
limited periods 
of availability. 
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TABLE 



STUJENT RESPONSE TO CAT IN CONJOINT COURSE 454 , 
LABORATORY PROCEDURES 



LECTURE CAI 6 PI PI 



Satisfied with material 

2 
3 



xos 17 5 

No 9 4 



Satisfied with method 

Yes 21 75 

No 15 44 



Future role of CAI, PI 
and Lecture 

Lecture 

More 0 00 

Less 9 22 



CAI 

More 



Less 



4 6 0 

0 0 0 



More 

Less 



6 6 4 

0 0 0 
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TABLE 4 



Dr. John Short 



MALIGNANT MELANOMA 



source 


tlf 


SS 


MS 


F 


t rea troents 


2 


29.435 


14.74 


0.916 


error 


63 


1013.682 


16.09 




total 


65 


1043.167 










N ° 66 







TABLE 5 Regional Medical Program 

RENAL FUNCTION IN PREGNANCY 



source 


df 


SS 


MS 


F 


treatments 


1 


.05 


.050 


0.013 


error 


18 


64.50 


3.586 




total 


19 


64.55 










N - 20 
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comparison or quality , preference and attention value 

BETWEEN THREE MODES 01 INSTRUCTION A 5 SEEN BY 
S».COND-YEAK MEDICAL STUDENTS 



Modes of Overall Quality 

Instruction Ke.m t-ratio 



CAX 


(C) 


2.44 


C-P 


1.30 


PI 


(P) 


2.34 


C-L 


2.96** 


Lecture 


(L) 


3.00 


P- L 


2 . 14** 





Preference 




Attention 


Kean 




t- ratio 


Mean 




t-ra tio 


1.8 


C-P 


1,86 


2.64 


C-P 


.24 


2.1 


C-L 


2.61** 


2.71 


C-P 


1.08 


2.2 


P-L 


.30 


2.90 


P-L 


.93 



* Mean computed across all students. 
** p < .05 



TABLE 7 



comparison or studi :;t preference for three : 

cr INSTRUCTION IN EACH OK TI!F. THREE 
INSTRUCTIONAL CROUPS 





r 


Modes of 
Instruction 




CAI CROlT 




PI TROUP 


• u; 


r-p.orr 


Mean 


t- r at in 


Mean 


t-rat 1 o 




t- ratio 


CAT 


(C) 


2.0 


C-P ,0633 


1.4 


C-r 2. 977- 


’ -r 


1.776 


PI 


(P) 


1.7 


C-L .3313 


2,4 


C-L 2.524-* 


L 


. 3 o i l 


Lee ture 


0.) 


2.2 


T-l 1.444 


2,3 


P-L .1407 


T 


.770 
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In December of 1*305, a co;:l: : i Ltec of j » physicians In the* isolated area of 
Control 'Mir,h ingLon J ■: t * t to discuss the need and opportunities for continuir a idne.it ion 
of physicians and other health professionals in that area. After a number of neet ings , 
tills group initiated and completed a comprehensive survey of nodical facilities and 
men power resources in Central h’nshugton. Most of the 240 physicians in the area par- 
ticipated in an attempt to identify their own conti no inn; education needs. 

These 240 physicians serve approximately 250,000 people* and an isolated from 
any academic and tied: cal facility by as much as 100 milts. There are nine small hos- 
pitals in communities within a radius of 100 miles of Yakima, the najor city in this 
Central Washington area, whose staffs confer with the specialists in the v akinn medi- 
cal community as frequently as time and distance will permit. Ko najor professional 
society meetings are held In this area and physicians who wish to attend continuing 
education courses must travel 150 nilcs or more to Seattle, Portland or SpoRaue. 

The content of the program tool; several forms, including exchange of teachers 
and practitioners between Seattle and Yakima, audio-visual programs, and the use of 
educational television- Visitation teams comprised of two physicians from the Seattle 
area spent two days in the city of Yakima in continuing education activities th.it were 
primarily directed toward physicians. Various topics determined by the Central Wash- 
ing! on continuing education committees have, been televised in a series of three shows, 
followed by a fourth meeting at which the instructors who made the television tapes 
visited Yakima for a two-hour wrap-up session, during, which a quest l on-and- answer time 
provided opportunities for discussing pertinent point. s in the televised programs. 

The ovcr-al] purpose tf the project Is to provide members of the Central Wash- 
ington medical communities with opportunities for teaching and learning in order to 
improve tne care for victims of heart, cancer, stroke rnd related diseases. 



CWJKCriVbS: 



file specific objectives of the project arc as follows: 

1* To provide opportunities for teaching and learning which will improve 
care of victims of heart, cancer, stioke and related diseases* 

2. To Improve communications between the Seattle academic complex and the 
Yakima complex which will augment continuing education* 

3. To improve communication between Yakima and the surrounding com nuni t l os 
in ways which will augment continuing education. 

4. To stimulate health practitioners to identify their ov;u erluca t ionai 
needs , 



3. To investigate records or whatever pertinent data are available to sub- 
stantiate needs or determine others. 



(• jo utilise pvi.ionnel r.nJ fuel lilies to gain maximum profit from hospital 
visit: by SluilIo .ioade. .lc personnel. 

7* To design evaluation i nst nrient s which will r veal prac t i l i oner ’ s piogivss 
toward goa J a . 
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To recreate habit s of self-inst rue t ton . 



9. To provide latest advances in the diagnosis, treatment and rehabilita- 
tion of victims of heart, cancer anu stroke in the Central Washington 
area . 



EVALUATION DESIGN: 



The Office of Ke search in Medical Education at tile University of h T a shinty on 
School of Medicine vns asked to .assist in the evaluation of this project. The first 
step was to identify the objectives of the program. These are listed above. Sec- 
ondly, our staff proposed an evaluation scheme for measuring the degree to which 
these objectives were achieved. Sonic of the objectives had been accomplished prior 
to this invitation. Also, it was not possible, within the limitations of time and 
funds, to evaluate *o:ao of f he project objectives. 

The following is an outline of the evaluation proposal: 



1 . Attitude Chan ge Men sure 

A semantic differential was developed to assess any change of physician 
attitude during the coarse of study. Examples of relevant concepts to 
be used in this study were: televised Learning, television, lecture, 

televised lecture, review material, research material, continuing medi- 
cal education, and practical material. The research hypothesis was 
that there would be a significant change of physician attitude during 
this course of study. 

2. Knowled ge Measure 

Three different designs were proposed to measure the amount of knowledge 
gained by participants who viewed the televised series. A pretest would 
be give; to identify entry knowledge and a post-test was to be given at 
least 30 days after the last televised series in order to assess long- 
term gain in knowledge, 

3. Change in Practice Measure 



The hospital records were to be examined prior to the series for patients 
whose final diagnosis for the admitting symptoms was known* Histories, 
tests and treatments ordered were to be recorded. The results were to bo 
assigned scares based on the criteria established by the lecturers. The 
scores bv the television series and after the series would be compared. 

It was p d that higher scores following the scries would indicate 

practice es. 



< + . Corvumir 



Eva 1 i it ion 



It was felt thit any increase:: in the frc;pn:u. ios of telephone calls, lit- 
ters, personal visits or referral letters between the physicians in Vaki a 
and select d rn die il prof ess Iona s in l h ' Seattle area eo.iid be interpreted 
as bein', related to the Central Washington Project. 
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1). Viewer 

A brief questionnaire was to be sent to viewers and potential v levers of 
these programs in ord«. v to assess i.he attendance am: attitude factors of 
those physicians from the Central Ua^nington area, 

A detailed proposed plan including the above areas was then presented to 
the Central Washington ccuni t toe (sec Appendix A). All of the plan was 
not accepted; for example , they felt it was r.ot the proper time to attempt 
to assess attitudes In the manner prescribed in t lie design. Tin; other 
phases of the plan were accepted with ror.ie modifications. With the ex- 
ception of the com:r:imica t ion phase, the notified plans were carried out. 
Evaluation data collection was adninistered by th: 3 Project Director and 
the results were analyzed by the Office of Research in Medical Education, 



Extreme care must be taken in interpreting the evaluation data from this project. 
The maximum return of evaluation forms was about 25 per cent of the group while, in 
most cases, it dropped to well below 20 per cent. There is every reason to suspect 
that people who did return their forms were aty pical . It is, therefore, misleading 
to use such an inadequate sample to generalize about the attitude of the total group. 
In fact, due to the small sample, the correlational analysis is not included in this 
summary. The following comments, then, apply only to those people who returned the 
evaluation forms. 

In general, the ratings Indicated a good, positive re/iotion to th'i experience 
on the part of all participants. There were no negative ratings for a ay session. 

The mate'ial presented was highly relevant for the participating doctors, as indi- 
cated by the high relevance score (3.13 out of a maximum of 4.00) and the fact that 
all expected to see some patients related to the presentation, and most (80 per cent) 
expected Lo see at least six cases per year for which the material presented was 
relevant. The nurses 1 sessions, although not so highly rated, also seemed to fill 
a need. The average relevance rating for nurses was 2,8 and, while 22 per cent 
reported that they were unlikely to see patients related to the material, 57 per cent 
felt that they would see more than five patients per year for whom the material 
would be relevant. 

The lowest ratings were assigned to the "suited to objective" item. Since no 
objectives were specified, this is not unexpected: when in doubt, people tend to 

use the center of a rating scale. However, the concomitantly lower ratings assigned 
to "interesting" and "Innovative 11 nay indicate that further attention to method of 
presentation and more stress on the purposes of a session would be worthwhile. 
Nevertheless, the total picture seems to indicate that participant:; wore well satis- 
fied--ns indicated by the over-nil roan of 2,5 assigned to "objectives* fulfilled" 
by both doctors and nurses. 

Recti .;neiul.i l i mv, : 

1, Earlier inv.^lvi.vnt in planning t Ik evaluitiou procedure ■; 1 necessary; 
especially, where Inf o i mat ion (such as sossic.i object iv- :;) is needed 
from the speakers. 



JJkSCRI i ; h ION AND RI.SULfA: 



Summary of the Fi rst Vis i tat ion 





2. The speakers should be Involved in the evaluation plans in order to 
elicit their cooperation and assistance in both Instruments 1 develop- 
ment and administrative procedures. It is imperative that they know 
and accept the purpose Cor the evaluation. 

J. Plans for collecting data must be more carefully worked out. It is 
rather useless to analyze the results unless at least 80 per cent of 
the participants respond. 

4. 'lucre is sufficient evidence to support the continued use of these 
evaluation instruments or nod ifica lion of the sane for gaining infor- 
mation of the type they purport to measure. 

5. Consideration should be given to the following two additional ways of 
evaluating these activities: 

a. Use of personal interview. 

b. Use of a group (M.D.'s, nurses, educator) to evaluate the 
program. 



Stro ke Yc lee nst s 
KNOWLEDGE GAIN h VALUATION 

A 25-item mult iplo-choicc test was designed by Drs. Fryer and fitus covering 
the material presented in tho throe televised stroke presentations and pretested on 
a group of physicians in a community similar to that of Yakima and its surrounding 
area, based on location from major medical facilities and size of tow . The test 
was then administered to 55 physicians in the Yakima area who had agreed to partici- 
pate in this aspect of the study. Five weeks after the Just presentation of the 
series, the physicians were again administered the same test. Unfortunately, only 
27, or 50 per cent, of the physicians completed the post- test due to various reasons, 
which included not having attended one or more of the televised sessions. All of the 
tests were administered anonymously, using codes, and results have been made available 
pre- and post-, to the participants strictly on an anonymous basis. Generally, in- 
terpretations made from t he data on this project should be guarded. The 50 per cent 

who took the post-test nay, in some manner, be atypical of the whole group. For 

instance, of the physicians returning the post- test, 85 per cent attended three or 
four of the sessions while those who did not take the post-test tended to attend 
two or loss of the sessions. However, there was no mean score difference between 
those physicians who did complete the post -test and those vho did not complete the 
post- test in terns of tho pretest mean scores. 

R esu lts: The test retest correlation coefficient for the instrument was .81 for 

those physicians who completed ‘noth pro- and post-test. This is a proLLy 
good indication that the instrument itself is a fairly consistent .arranger of those 
people taking it. The pro-test score for those persons completing both pro- and po ;i 
test was 3.84 an ! tho post-test score r*.« an was which represents a mean score 

difference which is sign! f ic i:d at the .02 level (t=2.t»7). 

discus si or. : The question of practice effect is not of much concern in this ens*- ns 

the time interval between pro- and post- testing is rather extended over 
a t hroe-mont li period. While it is presu eJ tEi.it the increase in knowledge is due U 
^ ~ ■ I ic ipn t ion in the televised stroke series course, in reality a study sue', as 

rniprs not prove this* in a definitive way. There are eeri.iir.ly opporl un i 1 i ns f oi 



i 



other var inblos to hive* intervened and caused this increase. However, one r iji;ht say 
that par t ic tpn t ion in tlu* project is a**- ocia Led with an increase in knowledge that 
is significant. 



j 



Chan ge ini jV_a ct ic c Ilea sure 



in order to root the objective of designing instruments to reveal practitioner's 
progress toward finals through the use of hospital records* a study has been outlined 
which would examine patients 1 records before the onset of the stroke program and 
after the completion of the stroke program with an eye toward measuring any differ- 
ences in physician behavior in terms of diagnosing, test-ordering, history-gather Inc; 
and examination procedure. It was decided that a method which would facilitate this 
type of investigation would be the preparation by Ur.°. Fryer and Titus of a sample 
"ideal" check list a Minst which each case folder could be compared. With this in 
mini, a pilot study .as conducted during the fall by a medical student who went to 
Yakima and collected this data fror 13 cases (see Appendix IS)* his results, which 
are summarised in Ins letter, sho.. hat such data collection is feasible and the 
results are interpretable . Generally, the pilot study scens to indicate that on 
preprogram case reports roughly only one-third of the "ideal" information has been 
gathered. It is planned to progress with the study in full over the summer of 1969 
and a further report will be made of the results of that investigation at a future 
date . 



As part of the assessment of the Wnshuigton-Ataskn Regional Medical Program 
telecast series, "The bedside JJiagnoris of Stroke," the Office of Research in Medical 
Education was asked to make a brief inquiry to assess the attendance and attitude 
factors of those persons who viewed or had the potential to view the three broadcasts 
dealing with stroke. A more intensive investigation into these two factors was not 
seen as feasible by the program planners and, consequently, a post card was construc- 
ted which was felt to adequately assess the type of inform icion desired. The results 
of the r ata are summarised below, 

The total number of post cards sent out was 144, Returns were divided Into two 
groups: those persons having attended a final wrap-up session following the three 

telecasts and those persons not attending the final wrap-up session. 

Table 1 shows that a very low percentage of those vho did not attend the wrap -up 
session returned the post cards (2.3 per cent). Of those who attended the wrap-up 
session, EG per cent returned post cards. It is impossible, presently, to say whether 
those vho did not return the post card and who did not attend the wrap-up session 
viewed or did not view any of the three telecast s« On the other hand, those persons 
who did attend the wrap-up session and presumably viewed one or more of the telecasts 
seemed more motivated to answer the post card and, thus, most likely constitute a 
differ nl sample than the non -at t« ndees . Thun it is most appropriate to view* the 
data and slnt« no;Us made about these data in Luis report as applying to the samples 
of atiiii-biS nr. J non-nl LeiiGecs only and not to generalize to the larger population 
of duelers ii; this Central Washing. t mi nr si. 

One inti rest in, feature* of the results is that those poisons who did not attend 
the wrap-up session sec ud to view the pronr.ua; on Liu whole more at homo tunn in tin 



Viewer Response to tjia Bed si tie Ding n os 1 s of Stroke Eropram 
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hospit-T] (see Tables 1 am i 2). Perhaps ill this; means i.~. tii.it those who are more 

involved with tiie ho s i La 1 s or located elm. or to t h- * hospi la Is a ret more likely Uj 

view telecasts such an these in L ho hospital, Rut it could also nears that those who 
wore aoro not i va t cd (i.e. , attended Lao wrap-up session in addition to L ho three 
telecasts) were those types of persons who would tend to go to the hospital to view 
the telecast. Of a possible total viewing percentage, that is, the number of people 
respondin'' for each sample tines the number of programs (i.e., 70 persons X 3 programs 
210 possible total viewing), 17s total viewings were indicated, or a viewing ratio of 
d3 per cent. For those who attended the wrap-up session, the ratio was 91 per cent. 

Again, either those persons who seemed motivated to attend the wrap-up session tended 

to view more telecasts, or else viewing more telecasts may have created an interest 
in attending the wrap-up session; from the data available, one can only hypothesize. 

Look in;; at the percentages ii Table 2, there appears to be a slight trend on 
hospital viewing to decrease over t lie three programs while the home viewing percent- 
ages seem to remain rather stable, The former tendency would agree with other studies 
involving radio or television presentations to nodical audiences, one in kew York 
State and tnc other in Utah. The latter phenomenon would have to be compared with 
information gathered from subsequent telecast series for confirmation. 

The respondents were as. d to indicate whether or not the telecast had been a 
worthwhile experience for then (sec Table 3). A much higher percentage (92 per cent) 
of those who had attended the wrap-up, session felt that tl g telecast had been worth- 
while, Table h shows that, again, a much higher percentage of those who attended 
the wrap-up session seemed to be more involved in this case using the stroke pamphlet 
which was mailed to all doctors in the Yakima and surrounding area. 

Attached are the various comments made on the post cards with regard to the use 
of the stroke pamphlet and general comments about the telecasts. In general, the 
telecasts were viewed favorably and it was felt that they were both worthwhile and 
instructive. There seems to be some feeling that the programs should be shown more 
than once with better physical arrangements, and tint the sessions could bo more per- 
sonalized; but, overwhelmingly , the responses were enthusiastic. 

In summary, the programs were well-received, the stroke pamphlet was considered 
valuable, not only for the programs but in its own right, and those who were involved 
were very much involved, 



MULTIMEDIA KOLLO'k-ln 

Due to a delay In getting rinse 2 of the Central Washington Project underway on 
time, the Regional Medical Program substituted five "fill-in" shows dealing with 
"Haml, Voice and Lye in the Diagnosis of heart Disease”; "Multiple Trauma"; "Obstet- 
rical Emergencies"; "Face Pain"; and "Placebos." These programs were aired at the 
s an* tines and over the sare channel as the stroke telecasts. 

As these programs did not deal with the same subject natter, It was decided to 
compare views r response so far as possible as a forth- r indication of the reaction 
to belli t lose television presentations as a i;!l. medium and of th.> illative popularity 
of tiie stiOre pr ogr a .V, . I a invest T.gn t i ivg tain quest uni, v;o ain- aware that no conclu- 

sions can be drawn frm such comparison as to whether the one .series was move popular 
thin 1 'ie soc and. 

o 
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Approximate I y one north after conn l- -Lion of the five programs , double post cards 
were seat oat to a] 1 physicians in the Y.i'.in.i ami surrcunding area. 1 ! request ing mat 
they indicate' who tit: r they had watched any of the programs and, if so, how many; Liie 
extent to which they liked or dislike*'] 1 lie .series, usln:; a Likert scale rar.g i ng from 
"not at all” to "very mich"; whether they preferred a multi topic, uni topic or hud no 
preference. They wore also asked to make suggestions for future presentations. 

Kesul t s : An average of three of the five maliitopic xhows were watched by l j3 physi- 

cians. Physicians who pave reasons for belay, unable to vat eh were not 
include..!. iJinety-onu physicians rated the over-all series on the like-Ji slike dimen- 
sion, The average rating was 4.4y out of a possible 5> The preferences wore indica- 
ted as follows: .lulti topic , 27; Uni topic, 33; ,lo Prefere’C-e, 33. 

Discuss ion : The results seen to indicate a very popular and oui^e favorable response 

for else multi topic presentation. It should be noted that nowhere near 
the amount of publicity that accompanied the stroke presentations occurred with t lie 
ihj 1 t iLopic series. kach individual presentation was not rated because such informa- 
tion was not of interest at this time. The rating with regard to the series is 
obviously very positive. It would be desirable to see whether the preference for a 
multi topic or uni topic or no preference was significant. A chi-square test was uti- 
lized with the result that the chi-square was not significant at the .03 level. 

While one cap say that some people- have no preference for either type of series 
and that some people do have preferences because of the actual indication (i.c,, 27 
people prefer multi topic , 33 people prefer unitopic), it is not possible to say that 
significantly more people prefer one cr the other nor that significantly more people 
have no preference. Perhaps a study is nccess ry attempting to determine what fac- 
tors influence the liking and disliking of either type of presentation. It nay be 
Mial there ate other factors that related to whether the topics were hultitopic or 
uni topic which have influenced those results. Another observation is that whether 
the respondents preferred r.ultitoplc, unitopic or had no preference, their ratings 
for the series based on present data wore very positive. Thus, it is not possible 
to say that preference goes along with like in this case. 

S ugges tions : Suggestions made by the physicians were not numerous and centered around 

more convenient schcduling--usually later in the day, surprisingly--ani 
around specific areas such as diagnosis of fractures, differential diagnosis and dys- 
lexia, dermatology, chronic respiratory disease, infectious disease, current status of 
antibiotic therapy, and use and abuse of laboratory tests. Kut also, there were sug- 
gestions for wrap-up session ; at the end of each series and re-runs of the series so 
that those who r.issed parts of them could view them. 

So; nary : There are, of course. Innumerable untold variables operating so that a study 

sue, i as tills cannot be conclusive in terms of causc-and-uf f eel relationships 
or even that the relationships that exist are not due to so;r.e unknown factor. for 
instance, it Is not known why tin rxiltitopie programs wore likoJ so well; only that 
they wre liked. Kor that matter, t ho stroke programs were also liked although not 
ns well, apparently, as the i:wil t itej i c ; yet, there is no preference for :vdlt j top ic 
over uniiopic, such as the stroke s*-rl ■ indicating n need for a fartinr study into 
the c nii' ; l L i v«; factors for likes ur.d dislikes and preferences. Kor idnti v<t lensen, 
the viltitopie pro ri a vre very us- 1 ) - 1 i ] f .\ , as well as we 1 1-va t ehed , (h-msajlv, :;u ;■ 
gen t ions for future pres n t U i < vr; v.vro in-dividual, ludleilin: that, ns far as Lais 
mens ufo in co;k* e.ria *1 , at this Lis: Lh- iv i ; no consensus is to want subject i .titer 
should be l phaslred in the future, 
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i)i scu-;. 1 ; j im: While Liu? full impact of tliis program cannot be assessed without tin* 

co; i;j J<- 1 ion of toe* chance in practice n. ;isur-? that will take place later 
in the* year l‘J6 f J, i nf ormul Lor* gathered to hate su?; ge its that tile program:; , as a whole, 
wort? successful in that a majority of tan viewers ratin' then positively. Tin* number 
of v fevers increased from the first presentation of the stroke series through the 
nul C media pro scuta t ions . ‘"net her tills was due to the content of the following pro- 
grams or whether it was due to an increasing interest in the televised C : HI broadcasts 
is, of course, not discernible* lint, it is possible, to arena that CMk televised 
instruction such as that presented in the Central Washington Project is in part justi- 
fied on tlu? basis of the interest shown by the physicians in that area over a wide 
variety of subject natter areas* Fur thernoro , consents from the physicians tiu-mselves 
in.icate that, not only did they increasingly enjoy the presentations, but wanted more 
There were many suggest ions for future broadcasts, not necessarily center in;* around 
the central RAV goals of heart, cancer and stroke. One thin;; tliat Is also perhaps 
apparent is tliaL it is probably not feasible to visualize any given physician viewing 
an entire series of a program operated in a voluntary manner* The results of the 
study indicate tbu.. Lhis was a rare occurrence and probably due to irregular aad 
heavy schedules tit it doctors in practice are subject to. 

As expressed, it was demonstrated that knowledge gain of a content type did take 
place over the stroke series. Whether the method utilized, namely televised instruc- 
tion, was more efficient than live lecture method was not investigated; other studies 
in tins area (research on teaching the visual arts, Jerome llausman, Ohio State Univer- 
sity in Handbook of dp. search oij Teaching . A. L. Gaga (ed . ) , American educational 
Research Association, Rand-McRally 6 Company, Chicago, 1967) have shown that tiier a is 
little apparent differences In terms of measures such as know! Jgc gained when compari 
sons are made between televised instruction and t ra lilional lecture methods. 

As the ultimate goal of the Central Washing; on Project was to effect a change in 
practice measure, the record study will he of cru« 1 . 1 1 importance. It is an attempt 
to quantitatively assess behavioral changer* at a behavioral level. Knowledge gain, 
enthusiasm of participants, and other non-behavJoi il changes are of little value utiles 
used by the physician in the diagnosis and treatment of his patients. 
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TABLE 1 



LOCATION AND VIEWING SUMMARY FOR STROKE TELECASTS 



■ — — r 

i 






Did Not 






Viewing 


Attended 


Attend 


Totals 


Program ; 


Location 


Wrap-up 


Wrap-up 


n \ 

1 

i 


Horae 

Hospital 

Viewers 


8 

35 

43 


9 

11 

20 


17 

46 

63 


ii 

■1 

n \ 

| 


Home 

Hospital 

Viewers 


12 

30 

42 


4 

9 

13 


16 

39 

55' 


03 


Home 

Hospital 

Viewers 


14 

30 

44 


6 

6 

12 


20 

36 

56 


Vicvii., 

Totals 

(all 


Home 

Hospital 


34 

95 

129 


19 

26 

45 


1 53 

121 
174 


programs) 











Total 

Returns 




47 


23 


70 




H 



TABLE 2 



LOCATION AND VIEWING SUMMARY OF STROKE TELECASTS 



Program 


Viewing 
Locat ion 


Attended j 

Wrap-up 


Did Not 
Attend 


01 


Hone 

Hospital 


19% 

81% 


45% 

55% 


n 


Home 

Hospital 


29% 

71% 


31% 

69% 


#3 


Horae 

Hospital 


32% 

68% 


1 

o o 


Viewing 

Totals 


Home 

Hospital 


26% 

74% 


, W 

58% 


Returns 




86% 


i 

ro 

v-n 

H 
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TABLE 3 



"Have th*se telecasts been a worthwhile experience for you?" 



Yes No Undecided 



Attended wrap-up 92 X 4% 4% 

Did not attend 62% 24% 14% 



TABLE 4 



"Did you use the stroke pamphlet? 11 



Yes No 



Attended wrap-up 96% 4% 

Did not attend 37% 43% 
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COMMENTS OP PARTICIPANTS ATTENDING THE WRAT-UP SESSION 

FOR 

YAKIMA STROKE PROGRAMS 
(N - 47) 



ITEM 2: 

* 

Excellent, Rood, well-done, etc, (12) 

To be aeon at least twice. 

I learned. 

Regardless of interest area, they are opening local academic channels. 
Strongly advise they be continued in same fashion as now. 

Out of my spocialty, hut interesting. 

Not in ray field. (3) 

Long time since last comprehensive review. 

Difficult to view entire program. Suggest one , : lialf-Lour showing. 
Lacks involvement of the audience — too general a topic. 

ITEM 3: 

Good, excellent, etc. (12)* 

Will keep for review. (2) 

Nice medical didactics* 

This will become a part of ray files with comments added. 

Program would be lost without it. 

Appreciate its brevity and conciseness. 

Well-done, but too brief. 

Not so hcl 'j 1 as M 2", 

More detail on drawings* 

lacks rany details I would like to review and don*t remember from T.V* 
Could have more systematic arrangenent. 

T . am a pediatrician — did not get a pamphlet. 



ITEM 4: 

* 

Excellent, well-done, informative, etc. (31) 

Necessary for completion of yroject. 

Personalized the sessions. 

Summary of emergency care good. 

Excellent; night would be better. 

1 had to leave after 10 minutes. 

Coed, not enough time. 

Not enough tine* poor organization. 

Very good. Hard to hear loud speaker. Poor reproduction. 

Need better physical arrangements. 

So-so--not too much he^p. 

We need rv^rc help with treatment. 

Not dynamic enough. 

??? 

T.V. repetitive, personal comments (Q & A's) by doctors excellent. 




COMMENTS 0? PARTI Cl PANTS NOT ATTENDING THE WRAP-UP SESSIONS 

FOR 

YAKIMA STROKE PROGRAMS 
<N - 23) 



ITEM 2: 

Well-organized and informative. 

Worthwhile in regard to this particular subject. 

Outlined quickly Importance of several quick, ea9y steps in diagrams. 
Would think a brief review in one to two months would reinforce retention 
of material. 

Would like continuous program on one subject until complete. 

Well-done, though ngt very useful to me as a pediatrician. I found them 
Interesting. (3) 

Fair review. Not worth time and money spent by production. 

By a good stroke of fortune, I didn't see them. 

Forgot to watch two programs. 

Television reception was poor. 

ITEM 3: 

Well-done, worth having for reference. (2)* 

Adequate for material covered. 

Help fix salient points. 

No time. 

ITEM 4: 

Good; let's have more, etc. (3)* 

Desire more practical Information; e.g. , therapy relative to specific 
problems . 

Usefulness of cancer and heart programs would be debatable 1 as regards ny 
specialty — ophthalmology. 

Telecast 02 did not begin until 7:50 a.m. 0 

Local problems with reception. 

Continuing education program would benefit from long»tern aeries. 
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RECORD STUDY 



1. Purpose 

In the final stage of the assessment of the Wash ington/A 1 asha 
Regional Medical Program telecast series, "Bedside Diagnosis of Stroke,” 
Lite Office of Research in Medical Education was concerned with measur- 
ing the effect th.if the Regional Medical Program stroke series had upon 
the cam of s t roke pa t i on t s in Yak i na by eva 1 ua t i ng the hospital records 
bo tli before and after the series. 

A i preliminary step a feasibility study vas conducted, August, 

19( id. At that time Mr. Haigh Tex. a fourth-year medical student, inves- 
tigated a sample of fifteen records from St. Elizabeth hospital vising 
a check list devised hy Dr. David Fryer, neurologist and co-author of 
the video series. A complete summary is available in the Appendix. 
Essentially the pilot study resulted in minor suggestions for revisions 
of the check list and in confirming the feasibility of conducting a 
comic to evaluation. 

1 1 . Desi g n an d Me Uiod 

A pre-series, post-series matched group design was employed. 

All CVA coded files available at St. Elizabeth and Yakima Valley Memor- 
ial Hospitals for a period of six months prior to the series presentation, 
October, 1968, and six months post were used. The files were analyzed 
and the number and kinds of inform t ion found were recorded on a "stroke 
series check list" (see Appendix) by Mr. Darrell Mull of the Office of 
Research in Medical Education. 

The combined number of files was 180 with approximately equal 
numbers in the before- and after- series divisions. Inspection showed 
that 3$ physicians authored these files but, when a given physician's 
patients were matched to hin and divided into pro- and post-series 
groups, some had handled only a single CVA patient, either p''e- or post-, 
and so vote deleted. The resultant sample was 22 physicians matched 
with their patients, each with one or more patients in his pre- and 
post- group. 

The stroke evaluation check list was the sole instrument for i he 
medical file evaluation. The five major categories: diagnosis, 

present illness, examination, order sheets, and laboratory tests had 
a potential of a '’noted generally" rating, indicating that >nly gc.erat 
information was recorded with respect to that category. The 48 more 
specific subcategories could be rated "yes,” (lIhVo y’os positive or 
negative information recorded regarding following), or "no," (there 
was iku information recorded), or "unoM.i i nab K , " (for whatever reason). 
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There was also a blocked area for additional clarifying notes* Noted 
generally" and "yes” received a numerical value of one point while 
"no" and "unobtainable" received zero points. The t v;o sets of scores, 
pro- and post-group scores, for the 22 physicians wore obtained by 
summing and averaging the. points from the cheek, list, 

Deli_nu tntj onjs : Several limitations of this instrument should be noted 

hero. Perhaps the major problem v;as in "quantifying" 
physician recordings such as, "The patient's head and neck are within 
normal limits," Does this mean that there is no skull tenderness bruits 
or neck rigidity? he decided to require an exact reference to a speci- 
fic tseaie ( stibca l eg* ry) b.Tore t in diting a point; this issue still 
required so no judgment on tin part of the investigator. Secondly, thei e 
were a number of scales which were not applicable on a given case, e.g. , 
"uses oral contraception" for males, or given "writing" and "reading" 
tests for a comatose patient. The former proved to be no problem 
because contraceptive use was not referred to in any of the files and 
the latter again required a judgmental decision from the investigator. 

Ill, S tat is t ic a 1_ Op e r a t i on a n d R esults 

The pre-series scores were divided by hospital to see if check 
lists from one hospital yielded higher scores. A H-test was done 
betv:ec*n the means of the two hospitals and the 1.934 value proved to 
be not significant. 

The physicians' files from both hospitals were combined and a 
correlation was run between the total scores, tne sum scores for both 
major and subcategories, of the 22 pre- and post-sets. There was no 
consistent relationship between these pairs of scores, evidenced by 
the near zero correlation, Pearson r = ,00389, The means for the two 
groups were essentially identical and a _t_- measure of difference was *047, 

Pursuing t He notion that the number of television programs watched 
mignt be related to the scores obtained, correlations were made, between 
the number of programs viewed and pre- and post-scores. No significant 
relationship was found for the pre- scores (r - .28) but a negative 
correlation, significant at the .03 level, was found between the number 
of pregrams watched and post-series scores (r =■ -.49)! 

Finally, the "noted generally" scales for the five major categories 
wire examined , Means wore de jved for this scale only and the pro- ami 
p ; t-se r its evaluated for statistical difference. Again tie means had 
to be assumed to be essentially the same with a j^-vahn of I. Mb. A 
correlation was next run between Llie number of programs watched by tne 
physicians and their "noted generally" post-score wean. ihe coeifiewal 
was not significant (r = .HO* 
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IV- Conclusions 



It can bo concluded that the data does not support the hypothesis 
that the "bedside diagnosis of Strobe" series facilitated bettor cave 
of patients if chart recording behavior is our criterion for measuring 
increase in quality of care. Naturally this criterion is itself ques- 
tionable. Perhaps a significant ircproveirent in patient care, diagnosis, 
examination procedure, etc. did occur. All we can say is tha . it did not 
appear upon evaluation of the nodical records. It should be "ecalled, 
however, that preciseness and completeness of file* recording was not one 
of the stated objectives in the stroke series program. To expect a sig- 
nificant change in recording behavior nay be, then, a little unrealistic. 

Generally, the statistical procedures shoved that the extent of 
relationship between such variables as nunber of programs watched or pre- 
scores and post-scores was negligible. The exception was the relation- 
ship wherein t lie higher post-scores tended to go with fewer viewings of 
the stroke series programs. While several interesting hypotheses for 
this relationship might be generated, the most reasonable is that there 
were s imp 1 y chance-factors causing tins statistical signi ffcanco. 

The scales were not of concern individually, for the nos t part, but 
a few warrant comment. The final scale dealing with a judgment ns to 
the physician's apparent grasp of the patient's problem proved to be too 
nebulous or elusive for the reseat chcr to adequately rate. The unobtain- 
able rating column proved to he another problem area, which did not lend 
itself to reasonable assessment and proved to be of little value to the 
study. 




stroke; evaluation check list 



■iosp 



Age M or F: PRE or POST; Physician 



. .d Unob- 

• i lly Yes ilo tai nabl e 



DIAGNOSIS 

type of lesion 
site of lesion 
degree of severity 

PRESENT ILLNESS 
at rest or activity "T 
sudden or progressive! Type of 
presence of headache j Onset 
severity of headache (if present) 
site of headache (if present) 

use of oral contraceptives (female) 

PAST HISTORY 
strokes 
syncope 

transient ischemic episodes 

hypertension 

migraine 

seizures 

family occurrences 

EXAMINATION 

blood pressure both arms 

skull tenderness 

bruits in the neck 

retinal artery pressure measure 

neck rigidity 

presene of hemiparesis 

degree of hemiparesis (if present) 

pronations sign or leg drift sign (if heni present) 

presence o r sensory disturbance 

presence cf extinction 

loss of optokinetic nystagmus 

visual field defects 

grasp reflex 

aphasia . 

speech J 

writing i as applicable 

reading ( 

understanding the spokanc word J 




ORDER SHEETS A LA3 TESTS 
sedative prescribed 
blood augar 
area nitrogen 
skull X-rays 
arteriogram 
EEu 

brain scan 



salt 

H„0 

calorics 

PT (if indicated) 
lumbar punctures 
anti-coagulation 

apparent synthesis ot 
patient's proolc'. 



APPE1IDIX 
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C 1 LAh \j h M I vA S L jin 



Cons i d era t l. o n s i 



The Semantic Differential is a device for measuring attitudes toward a 
construct and also the relationships of the construct concepts to each 
other. The most commonly used scales define tho construct selected on 
the dimensions: evaluative (good-bad), potency (weak-strong), and 

activity (active-passive). 

Thus a concept such as "television*’ could be seen as strong , active, 
and potent or as various degrees of these factors. It could also be 
seen in relationship (i.c., where it stands with all other concepts 
being measured). The overall evaluation of the concepts defines an 
attitude toward the construct (aero, the Yakima Project). 

The purpose of the Differential is for attitude measurement and it may 
be used to test attitude hypotheses. It is suggested that one way of 
measuring the effectiveness of a continuing medical education program 
would be to measure attitudes toward relevant concepts of that program 
before and after the program. 

The following are examples of what are felt to be seme relevant con- 
cepts to continuing r.edical education attitude, specifically for the 
Yakima Project: televised learning, television, lecture, televised 

lecture, review material, research material, continuing medical educa- 
tion, practical material. 

The research hypothesis is that there will be a significant change of 
physician attitude during the coarse cf study: specif ically , that 

"televised lecture," "televised learning," and "continuing nodical 
education" will cone to be seen as more active, stronger, and better. 

We feel that it is also important to see whether participants* atti- 
tudes toward: "television, " "lecture," "review material," "research 

material," and "practical material" change as a result of these programs. 
Other studies, such as "Urology for Mon-Urologists," have shown disagree- 
ncr.t about these concepts and continuing medical education, The data 
would provide guidelines for the planning of future programs by serving 
to 1) identify the desires of the audience, 2) specify the composition 
of the audience with regard to their attitude toward the purposes and 
tho presentation of the program. 

The Semantic Differential is thus a flexible and demonstrably reliable 
device for r, ensuring attitudes. It is felt that attitudes are one indi- 
cation of the effectiveness of a program such as the Yakima Project. Tho 
scale is completed vapidly (approximately 15 minutes) and the data cm 
be dealt with anonymously. 
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DIRECTIONS 
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below is a term which can be rated by placing an X in the space which best 
represents your feeling about that tern for various pairs of words. 





House 




very 




very 



good ‘ | : : bad 

active : : : : : : passive 

weak ; ; : : strong 



First, you would decide whether you feel HOUSE seems "good 11 or ‘'bad." If 
you feel HOUSE was very "bad," you would place an X in that space. You nay 
place an X wherever you feel it should be to represent your feelings about the 
tern HOUSE. 

While this nay seen difficult, it has been shown that such ratings of terns 
are possible and reliable if done raoidly, without dwelling on each scale. It 
is then possible to get a picture of how each tern is seen with regard to the 
various pairs of words. 

On the following pages arc several terns. Please work rapidly and nark each 
tern on every scale. Do not omit any toms or ratings. 




In 



very 



very 



liflht 



bad 



larp,e 



clean 



fast 



weak 



usly 

passive 

sharp 



heavy 

gocd 

srrall 

dirty 

s low 

strong 



beautiful 



active 



dull 



f 




1 



To lev is ed learning 1st 



very 



very 



.large 
slow 
veal; 
beautiful 
active 
1 ight 
clean 
good 
sharp 



small 

fast 

strong 

usiy 

passive 

heavy 

dirty 

bad 

dull 







very 



fast 

qglv 



very 



slow 



beautiful 



large 

active 

strong 

dull 

dirty 

heavy 

good 



small 

passive 

weak 

sharp 

clean 

li?M 

bad 




V.K) 



T e* lev 1 «] icdjt ! c t ur e njj ire : 



very 



very 



f 09t 
passive 
heavy 
8TTia] 1 
dull 
good 
ugly 
strong 
clean 



8 lOV 

active 

light 

large 

sharp 

bad 

beautiful 

weak 

dirty 
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Revi ew : 



very 



very 



tgiy 

good 

large 

sharp 

clean 

li^hc 

weak 

slow 

passive 



beautiful 

bad 

snail 

dull 

dirty 

heavy 

strong 

fast 

active 
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Research material J.s: 



very 




very 


active : i ; : 


, 


: passive 


small : : : V 




: large 


fa3t : : '* '• 




: slow 


dull : : : : 




: sharp 


clean : : ; • 




: dirty 


bad : : s 




: good 


light : : : : 




j heavy 


weak : : : : 




: strong 


u&lv : : r : 




: beautiful 










Continuing Medical Education is: 





very 


very 


email 


• • • • * 


: large 


beautiful 




: ugly 


heavy 




: light 


sharp 




: cull 


strong 




: weak 


fast 




: slow 


good 




: bad 


active 




5 passive 



o 

ERIC 



Practical rvaterinl is: 





very 




very 


bad 


• » « i 


. . 


good 


uf>ly 






beautiful 


active 






passive 


heavy 






light 


weak 






strong 


dirty 






clean 


fast 






slow 


dull 






sharp 


large 


, 




email 
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II. GAIN NhASUuK 



Cons Id e_r.it ions : 

1, There mast he a base rate group which Joes not: attend which is 
tested before and after the sessions. If necessary, the com- 
parison could be made on two random groups from the population 
of non-par tic ipa ring physicians* 

2, There must be a pre- and a post-test demonstrably equivalent; 
or the sane test may be used if test -re test reliability is 
shown to be high. 

2, if the .sa.no group can be tested before an I after a particular 
session, or series if possible, then the data ca n be evaluated 
i n terms of net gain. 

4. Idea’, ly, the groups would be stratified samples; however, Ibis 
seems to present too many problems of data gathering. 

5. All designs are for one session or one series; they would have 
to be repeated for each area, e,g, f cancer, stroke, heart, if 
information is desired for each. 

6. Preliminary test construction should be done on two groups sep- 
arate from those who attend and those who do not attend samples. 

7. Whether the physician watched at home or at a hospital, what 
type of practice he engages in, age, date of graduation, and 
postgraduate study should be considered in evaluation. Thus, a 
third design would be analysis of covariance or randomized blocks, 

8. The period after the sessions before post-testing determines 
whether we ar> measuring immediate recall or permanent knowledge 
gain, Thus, a decision must be made for when post-testing will 
take place, 



DESIGN’ 1: 

A random sample of the non-par ticipat ing physicians for any given session, 
or series if possible, is to be administered test .a. A random sample of 
die participating physicians for the same session or topic will also 
receive test a prior to the session, After each session, a random sample 
of participating physicians, as well as a random sample of non-participa- 
ting physicians, will take test b>, {Note; The same design would apply 
for a test retest using the same instrument.) 

A random sample of non-participating physicians for a given session (or 
series) will receive test <j. A random sample of participating physicians 
would also receive test Both groups would be tested at the end of the 
session or topic . 







d;:sig:c :i: 



lmi on n , 0 , postgraduate work, viewing conditions, practice, and date of 
E lation would be gathered for all physician, participating, in the study. 
Pee- and nose-tests would be administered to those physicians attending a 
liven session (or series). Also, a control group would be measured and 
demographic data gathered. A randomized block design would be used to eva 

uate the results. 



Design 2 is the most preferable, desif;n 
is least preferred from a statistically 



2 is next preferred, a*sd design 1 
defensible viewpoint. 
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Ill, 



change in iVvAcrici; nkasui;:: 



Co nsid era t ions : 



1, Cooperation for access to records would be necessary from i he 
hospitals serving the central area. 

2, Criteria for desired diagnostic tests and tr eat men ts v uld have 
to be established. 

3, It. would be necessary to knov; which doctors attend*. 1 cc h 

lecture. 



4. A staif would be necessary to uaarch the records. 

*>. N'c knowledge testing -ould be necessary to evaluate the of dec-* 
tiveness of the project. 

DKS70N : 



hospital records be examined for a period prior to tut for 

patients whose final diagnosis for t admitting symptoms is known. 
Histories, tests, and treatments ordered would ba recorded, TJie results 
cou:d be assigned scores based on criteria established by the lecturers. 



In order to derive criteria for assigning scores, it would be necessary to 
neet with the lecturer (s) for each program. Minimally acceptable practice 
and desired practice levels as well as the relative importance (weights) of 
the various parts (i.c., history, treatment, and tests, etc.) would have to 
be ascertained. 



The first examination of the .ospital records could begin rs soon as the 
final ; rogiam had been written and the criteria established. 

At a determined period after the series, the records would again bo exam** 
ir.ed using the sane, criteria. The scores before the series fnd after the 
series voula then oe compared. Higher scores following the series would 
indicate practice changes. 

The relationship between ninber of lectures attended and practice change 
could be examined as well as other relationships determined in advance 
provided the ;ata are nccesoible. 



i v * io:; uvai.uatio:: 

Objectives 2 and 3 for the Yakima Project arc to improve cnmr, unicat ion 
between Seattle and Yakima and Yakima and its surrounJing communi t ios , 

Vo feel that four types of measurable comma ni cat ion are: 



1. telephone. 

2. letters. 

3. personal visits. 

U , referral letters . 

Increases in frequency of these com-runico rion channels could be inter- 
preted as bcin;.; related to the Yakima Project if a base rate count were 
established before the series for comparative purposes. 

Me thod : 

1. Select a sample of physicians in the Seattle and Yakima areas who 
are referral targets. A sample would be selected for each se: ica 
and would include the series lecturer(s). 

2. If cooperation can be obtained > the secretaries would record (count) 
the number of contacts for each of the four above methods of communi- 
cation, This count would start inwee ..ately and continue for three 
months following the particular series, 

3. Recording forms and instructions will be supplied to the secretaries 
by the Office of Research in Medical Education. 
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hi he i pa res i s L.ic ,* : er s:oc 1 oi do..v.s t n j ;.{ nuncio 

(a) is | ;rvil yxe .1 cq the opposite si co , 

(b) is pure 1 y,p’d only the t rnpezi nr. inu-cle is 

invoJvi d. 



V ]■ 






C c- ) ray hi- ' y ,:ed without i nvo3 venont oj she trapezius. 



T V 



( tl ) is new r purely zed . 



T K 
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IscluT-; i o at. L «k 1 : s 








(.i) 


are nearly always followed 


by a completed stroke. 


r 


Y 


(b) 


a re ,u s . ► 5 . at e d w i t h a v t e r i 


al stenosis. 


T 


Y 


(c) 


urai/i 1 1 y ^et progress] vo ly 


i. ore severe. 
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(d) 


are f.airrd either by spasm 


or s>ii»all on.be 1 i . 


T 


Y 



6 , / p u re trio L c r jj t r < 'he : s eh - * ra c t o r i s t I ca ] ] y 

(a) caused by occlusion of a snail penetrating artery. T Y 

(b) caused by a lesion in the motor cortical area. T F 

(c) associated with carotid artery disease ciu! low blood 

pressure, r * T Y 

(d) caused by a lesion in the basal ganglia. T Y 



7, Apraxia 



(.->) 


is 


of no localizing value. 
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(b) 


my 


■ involve on, ly the left hand. 
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(c) 


i s 


caused by ceiebellar disease. 




r 
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(d> 


i s 


caused by a con hi tied loss of sensation. 


slvongt h , 


Y 


V 



and coordination. 




8. A grasp reflex is 

(a) a sig/u of a lotion in the frontal lobe on the 

oppoc- j t o .side. T F 

(b) elicited by f i ria pressure on Llic back of the band. T F 

(c) an early sign of temporal lobe herniation. T F 

(d) most commonly seen in subarachnoi d hemorrhage. T F 

9. Spasm of cerebral arteries 

(a) gives rise to transient J sthenic attacks. T F 

(b) m ay occur in migraine attacks. T F 

(c) couurvonly causes completed strokes* T F 

(d) occurs with subarachnoid hemorrhage. T F 

10. Occlusion along the course of the Posterior Cerebral Artery 
character is t i cal 1 y 

(a) produces homonymous hemianopia . T F 

(b) is associated with abnormalities of optokinetic 

nystagmus. T V 

(c) produces an 4 nfarct in the medial occipital lobe. T F 

(d) is commonly not associated with symptoms othei than 

homonymous honianopia. T F 

11. Paralysis of the third cranial nerve prodi^ccs 

(a) severe ptosis. -TV 

(b) a large pupil T F 

(c) paralysis of upward and jiu.nrd gaze. T F 

(d) deviation of the eye outvnrtf. T F 
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3 2. In henipnresis , the muscles of the forehead are not usually 



3 2. In henipnresis , the muscles of the forehead are not usually 
involved because 

(a) they are supplied by both facial nerves. T F 

(b) they are involuntary muscles. T F 

(c) they arc supplied by both the 5th and the 7th nerves. T F 

(d) they are innervated by both sides of t he cerebral 

cortex, i" F 



33. Blood in the spinal fluid is 

(a) not caused by a traumatic tap if the centrifuged 

fluid Is xanthochromic . T F 

(b) always present in intracerebral hemorrhage . T F 

(c) always present 5r. subarachnoid hemorrhage. ,p F 

(d) usually absent in cases where the spinal fluid 



pressure i? elevated. T F 

14. Neck rigidity is an important sign of 

(a) subdural hematoma. T F 

(b) subarachnoid hemorrhage. T F 

(c) Migraine. T F 

(d) basilar artery thrombosis, < T F 



15. Headache is 

(a) character ist leal ly present in cerebral embolus on 



the side where th» embolus is lodged. T F 

(b) characteristically severe in subdural hematoma, T F 

(c) often absent in subarachnoid hemorrhage. T K 

(d) often absent in intracerebral hemorrhage. T F 

74 



74 




^ ('1 



l J 

16. I'xtinclion of sensation is a sign 

(a) of a lesion in the thalamus. 

(b) of prognostic innortance, 

(c) of disease in the parietal lobe;, 

(d) characLoi: isticallv associated with a grasp reflex. 
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t r 



17, The corpus call osina is 

C2-7 (a) a large, structure connecting the cerebral heni pheres. T F 

(b) supplied partly by the middle cerebral artery. T F 

(c) supplied chiefly by the anterior cerebral arLery, T F 

(d) supplied partly by snail penetrating arteries from 

the circle of Willis. T F 



IS. A lesion in the temporal lobe characteristically 

(a) produces abnormalities of optokinetic nystagmus. T F 

(b) produces superior homonymous quadrant inopia. T F 

(c) is caused by occlusion of the anterior cerebral artery. T F 

(d) may damage the lower part of the optic radiation. T F 



19. Paralysis of the 3rd cranial nerve caused by diabetes is usually 
character lzod by 

(a) absence of ptosis. 

(b) the pupil being smaller than on the normal side. 

(c) absence of pupillary involvement. 

(d) bi la t oral ity , 

: 

i 



T F 
T F 
T ¥ 
T F 
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(c) 


often bilateral. 
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associated with nystagmus in one eye. 
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In men: uremcnL of retinal artery pressure 
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the examine! observes the arteries onl) in the optic 
di SC LI , 
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the onset of arler ? al pulsation as the pressure is 
increased indicates systolic pressure. 
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pressure is applied over the anesthetized cornea. 
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(d) 


Measure men l must. be made on both sides. 
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St i okc-i n- cvol u Li on 
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is associated with migraine. 
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F 




(b) 


is characteristically associated with hypertension. 
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(c) 


is characteristically followed by good recovery. 
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(d) 


may be caused by a cerebral hemorrhage. 
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23. 


Interne] 


carotid artery stenosis may be indicated by 
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reduced retinal artery pressure on the affected side. 
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(b) 


lowered blood pressure in the arm on the affected side. 
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neck bruit. 
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Amaurosis IV gay. 
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7 This tost it? anonymous. however, in order to evaluate this program, 

it is necessary to have sc<:m means of keeping the results separate. 
For this reason, please wriLe t ho month and day of your birth and 
your middle initial on the lines below: 

Month ; ; 21 

Day 

Initial ; 

PhKASK check to make sure that you have answered each p n t of each 
question and have filled in the persona] information requested above. 
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Charles Dohnor, Fh.J). 
Office of Research and 
Medical Fduculion 
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INTRODUCTION 



Recently trie President of the Washington State Medical Association, 

Charles D. Muller, M.D , wrote in part: 

"Adequate health care of the highest quality for all Americans 
has been moved from the realm of debatable social issue to a 
position of public policy. . . ^ 

The medical profession has promoted this quality by several means. One 
is by providing opportunities for continuing education for its members. 

Licensure laws were enacted before the technological explosion which 
began, in fhe late 1930's. Asa result, these Jaws did not recognize that nsw informa- 
tion and innovation would render a licensee's original qualifications obsolete unless 
periodically upgraded. Few stote laws regulating medicine require the prevention 
of professional obsolescence. But the physician knowj that he must keep up because, 
as Dr. Muller points cut, "The medicine of today and tomorrow is different from any 
ever practiced before. It can do enormous good, but ;s also dangerous and ran harm . 
At one extreme, contemporary medicine is so complex that a single physician cannot 
provide it with safety and efficiency. At the other extreme, it is so simple that a 
remedy can be applied by anyone of overoge ability . c 

The needs and the demands of the phy Icinns for continuing medical 
education in recent years have been provided in part by the University of Washington, 
the county medical societies, hospitals, the Cancer Society, the Heart Association, 

1. Mul'er, Charles D, , M.D. , President's Page, "Quality of Health Care and 
Licensure' 1 , Northwest Medicine, August 1968, p. 769, 

2. Ibid. ' 



specialty societies, the Regional Medical Program, and the physicians' own State 
Assoc iation , 

Seminars, symposia, circuit courses, grand rounds, consultations, and 
extensive postgraduate courses are all used as means of transmitting this medico! know- 
ledge. And in the interim, the physicians read journals, newspopers, books, and 
listen to the radio and recorded tapes, and wotch medical television in an attempt to 
"keep up". 

How well are the continuing medical education needs of the physician 
being met? Are the physicians satisfied? Can they find worthwhile courses in the 
state or do they have to go elsewhere? What about costs and time and convenience 
of location? Is one type of program, one sponsor, one media better or worse than 
others? On what subjects would the physicians like to have more emphasis placed? 
T hese are pertinent questions to those who purport to serve the physician, Physicians, 
as a group, are very astute, wonderfully dedicated and motivated, and busy servants 
of mankind. The physician needs help, the right kind of help, and he is tf e one to 
ask how well we are doing by him. 

Thus the Regional Medical Program and the Washington State Medical 
Association, through its research arm, the Washington State Medical Education and 
Research Foundation, decided to pose these questions To the physician popilation of 
the Stole of Washington. 

From November of 1966 to January of 1968, Mr. Richard F. Gorman, 

Di s . Lucius 0, Hill, Donal R. Sparkman, John R. Hogness and others wrestled with |he 
problem of Iwv/ to conduct the survey and how to finonce it. Their original plans were 
rather ambitious -- very complete and well considered -- but financing become too 
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much o problem so the plans were modified and cu) down so that the Regional Medical 
Program could underwrite much of the cost. 

By January of 1968 the plans were complete, the financing was allocated, 
the survey was pre-tested, all the ground work was finished, so the questionnaire with 
which to gather the necessary information was turned over to the survey staff. 

Seventy-five percent of the questionnaires mailed were returned and it was 
thought to be quite good under the circumstances. The 4,778 in the survey included all 
physicians with Washington State addresses registered with the American Medical 
Association. This included Washington State Medical Association members and non- 
members, interns, those in residency, teachers, administrators, those in the military, 
other government service, retired, private practice, some were ill and some had died. 

This study is based on the findings from 3,364 questionnaires which were filled 
in (not necessarily in their entirety) and returned prior to the cut-off date of July 18. 

In summary, here is the breakdown of the questionnaires: 



QUESTIONNAIRES MAILED, RETURNED, AND USED 



Total Mailed 




4,778 


Total Returned 




3,604 


Percent Returned 




75.34 


Total Returned 




3,604 


Not Used in Survey 


Returned too late 


37 




Returned without name 


O 




Returned but not used 


for other reasons * 


200 


240 


Total Used in Survey 




3,364 



*fhysician wrote that he did not feel qualified because ne was retired, not in 
private practice (teaching, in armed services, intern, in residency), or in 
the case of six physicians, their wives wrote that the physicians were 
deceased. 







CONCLUSION 



The completion of this survey on the needs ot the physician for conlinulng 
medical education leaves the project directors with these conclusions: 

1. Those responsible for providing continuing medical education fcr the 
physicians in the State of Washington are performing their task well. The quality of 
courses and formal programs, however/ is rated much higher by the physician popula- 
tion than is the quantity. More needs to be done. 

2. Specialty societies are doing much better with their programs and 
journals than any other group. This suggests that the other groups may well examine 
what the specialty groups are doing so well to please the physician. Perhaps this 
examination will show only i ha t comparisons should not be made, for the audience 
and the purposes are not the same. Broad in one ca i ;e u.nd specialized in the other, 
perhaps the meetings rating the lowest are still proving their w^rth and cannot be 
expected to be more effective in the area of continuing medicol education. Likewise, 
media such as Tel-E-Med, educational television and motion pictures, which were 
rated so poorly by the physicians, may be doing all that reasonably can be expected 

of rhem . It is entirely possible, however, that a more Imaginative director may produce 
television and motion pictures which the phy'icians may rate much differently. 

3. Greater attention needs be paid to the continuing medical education 
of the rural general practitioner. 

4. Concerning the questionnaire itself, this conclusion was reached: 

The questionnaire was a good one and provided much interesting and 

infoiTTiotive data . As with most things, though, flaws did show up. Some of the onswers 
were difficult to evaluate and others raised further questions. Some of the information 



gained needs to be clarified and expanded upon or related to other pertinent informa- 
tion . The lost can be done by p atching the questionnaire results with certcln physician 
data, e.g, , hov/ did a particular specialty or age of physician answer which questions? 
This can be done by matching the questionnaires to the physician on file in the data 
bank of the Washington Slate Medical Education and Research Foundation through the 
physician's identification number. Several variables can be extracted from the data 
bank to cross -tabu fate with the results of the questionnaire. Clari tication of some of 
the answers rece /ed may best be done by personal interview or through another more 
specific questionnaire on a sompling basis. In any case, it is felt that more needs be 
done to make the results of this questionnaire as meaningful as practical . 
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SUMMARY HIGHLIGHTS 



1. 


Of those physicians who attend courses and formal programs of continuing 
medical education, 71% ore usually satisfied. 


2. 


Twenty-one percent of the physicians in the state attend continuing education 
programs too infrequently to render an opinion on the subject. 


3. 


For every critical comment on courses from those usually satisfied, ten comments 
of a critical nature were received from those who were only sometimes satisfied 
with those courses they had attended. 


4. 


Four out of ten (39%) critical comments stated that the courses were not practical. 


5. 


Almost two-thirds (58%) of the physicians can often find an interesting course 
to attend, one-third (33%) occasionally find an interesting course to attend, 
and only 8% rarely find an interesting course* to attend. 


6. 


Nearly half of the physicians say that the greatest deterrent to their attending 
courses is time, and it makes no difference whether the course is in state or ou* 
of state. (This certainly attests to the fact that we live in the jet age.) 


7. 


Specialty group meetings rank highest in attendance, followed closely by 
one to three day symposia. Extensive postgraduate courses are attended least. 


8. 


Specialty group meetings were ranked highest as to overall effectiveness, followed 
again by the one to three day symposia. County medical society meetings were 
rated as lec.it effective. 


9. 


Specialty journals were rated tops as a medium in ' keeping up". Tel-E-Med was 
n distant last, with medical educational television and motion pictures only 


O 
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slightly better . 
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10 . 



One out of three physicians had suggestions as to subjects which have not 
received sufficient attention in continuing medical education programs. The 
practical aspects of practicing medicine were thought to be slighted most. There 
also are a large number of doctors who think that more courses should be aimed 
specifically at the general practitioner. 

1 1 , f or the management of heart, cancer, and stroke, the greatest need, as the 
physicians see it for their own communities, is to eliminate the financial 
barriers to the availability of service, diagnostic facilities were thought to 
be the least needed. 
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RES JITS OF THE QUESTIONNAIRE 



QUESTION 1 

With i veil'd t<» i or it inning medical education ionic's and nIIht Iniinaj prunrani'' mi n nlh 
j lu im' made available to you. n milt s, r f fhrir ^/.i/uiwvdup. would you i*ive \ihii overall 
evaluation of tnc i It re (i\ vne>s of eoiiiM > and ullu r jr in rial ji r< i u;i.i 1 1 is » v checking our of the 
follow imj .statements; 

( heck Onv 11 1 am usually satisfic'd will) tl k I have atlriali’d. 

h I am only \oun ihtu s satisfied with these 1 li.iw attended. 

□ - 1 r/< *f 1 1 •itfi'nd then) ott< n enough to m.ih' an f\ aluatinu. 

Oiimnents: . . . . _ 



Answers to this fi*r$t question provide insight r'fo the degree of satisfaction 
which physic fans general ly obtain from having taken continuing med’cal educotion 
courses and how effective the physicians think these courses are. 

More than half (55%) of the physicians in the survey indicated that they were 
usually satisfied with those courses and formal programs they had attended. Twenty- 
two percent onswered that they v/ere only sometimes satisfied, arid 21% answered 
that they don't attend often enough to make an evaluation. 

Twenty-one percent of those who answe'ed Question \ also provided comments. 
Those who v/ere usually satisfi 2 d seemed to attach importance to selectivity . Eighty 
specff*ca!ly mentioned that they were selective in the courses they attended. Only 
fourteen v/ere critical of the programs they had attended. Eleven of the fourteen 
made the comment thut courses were tea didactic and three commented that they were 
too specialized. 

On the other hand, 124 of those who answered that they were only sometimes 
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satisfied lent criticism. The comments follow: 



Comments Number 



Not of practical value 54 

Too elementary 20 

Poorly prepared 10 

Too general 7 

Repetitious 7 

Too esoteric 6 

Poorly presented 6 

Stereotyped 6 

Too social 5 

Teachers often promoting selves 3 



Of the 689 (21%) who answered that they don f t attend often enough to make an 
evaluation, 164 made comment cs to why they do not atte.-d. These comments will 
be combined with the comments to Question 3 d, "Other reasons why you seldom 
attend such courses and programs . " 

From this group of physicians who said that they oidn't attend courses and pro- 
grams often enough to make an evaluation, some did go right ahead and evaluate parts 
of Questions 2 and 4. Following is the detail: 

Question 2 

"I often find an interesting course f c attend." 

6% said "In State" 

10% said "Out of State" 

19% said "Both In State and Out of State" 

"I occasionally find an interesting course to attend." 

1 1% said "In State r ' 

9% sc id "Out of State” 

18% said "Both In State and Out of State" 

"I rarely find a* interesting course to attend," 

7% said "If. State" 

2% said "Out of State" 

8% said "Both Ir State end Out of State" 
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Ques tion 4 (They were asked to leave b'ank any which they did not attend.) 



44% evaluated One to Three Day Symposia 

26% evaluated Circuit Courses 

50% evaluated Regular Recurring Hospital Piogiams 

How these physicians evalu r :ed the above continuing education media and how 

their evaluation compared with the evaluation of the physicicns who said they attend 

courses and formal programs is discussed in the section dealing with Question 4. 
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This question pertains to the aval lability of courses, in state and out of state , 

The question is somev/hat difficult to evaluate because the physicians were asked 
to check all statements applicable and os a consequence, the 3,364 who answered 
checked 3,645 statements. Of the 3,645 statements checked, 2, 124 (58%) were 



under 2 a, "I often find an interesting course to attend' 1 , 33% of the checks were 
under 2 b, "I occasionally find an interested course to attend 11 , and 8% were under 



2 c, "1 rarely find an interesting course to attend' 1 . 



A further breakdown of the answers follows: 




2 a. I often find an interesting course to attend: ( 2 , I24cn$wered) 
(3% answered “In State" 

19% answered "Out of State ,T 
67% answered "Both In State and Out jf State* 1 
10 

*) > 



2 b. I occ osion olly find an interesting course io attend: (1,213 answered) 

50% answered "In Stvte 1 ' 

16% ansv/ered "Out of State" 

54% ansv/ered "Both In State and Out of State" 

2 c. [rarely find an Interesting course to attend: (308 ansv/ered) 

43% anssvered "In State" 

11% answered "Cur of State" 

46% ansv/ered "Both tn State and Out of State" 

How Question 2 was answered depended somewhat on how Question 1 was ansv/ered 
A larger portion of those who were usually satisfied with courses they had attended 
often find interesting courses to attend than those who are only sometimes satisfied with 
courses they have attended. Of the latter group, a larger percentage only occasionally 
find an interesting course to attend than from among those who are usually saiisfied with 
those they have attended. Here is how the two group: evaluated Question 2: 



Ihose who in 
Question 1 stated 



I often find an interesting course to attend." 





that they were: 












In State 


Out of State 


Both In and Out of State 




Usually satisfied 


17% 


~ 17% 


52% 




Sometimes satisfied 


7% 


16 % 


34% 






,r l occasionally find an interesting course to attend." 






In State 


Out of State 


Both In and Out of State 


] 


Usually satisfied 


13 % 


7% “ 


16 % 


! 


Sometimes satisfied 


18 % 


11% 


34% 


i 




”1 rarely 


find an interesf : ng 


course to attend. " 


i 




In 5tate 


Out of State 


Both In and Out of State 


i 


Usually satisfied 




1% 


i% 




Sometimes satisfied 


7% 


1% 


2% 
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This question dealing with the barriers to attending courses was answered by 
2,610 physicians. The results: 

1,237 (47.39%) checks for "Time” 

372 (14.25%) checks for "Locale" 

435 (16.66%) checks for "Costs, such as" 

268 (10.26%) checks for 'Tuition" 

366 (14.02%) checks for 1 Other direct costs" 

They were invited to check one or more and they did provide 2,678 checks. 
Further programming on the computer will eliminate the checks for 3 c, which duplicate 
checks in 3 cl and 3 c2. 

The inconveniences itemized in this question which cause scanty altendance 
lo progroms and courses were cross-checked with the answers in Question 2 with these 
rc su Its: 

A. The percentage of those who said time was on inconvenience: 



38% of the 401 v ho ^aid they often find an mterestr j course to attend 
in state 

39% of the 508 who said they often find an interesting course to attend 
out of state 

44% of the 1,359 who said they often find an interesting course to 
attend both in and out of state 
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50% of the 460 who said they occasionally find an interesting course to 
attend in state 

43% of the 274 who said they occasionally find an interesting course to 
attend out of rtate 

54% of the 684 who said they occasionally find an interesting course to 
attend both In ond out of state 

37% of the 203 who said they rarely find an interesting course to attend 
in state 

43% of the 60 who said they rarely find an interesting course to attend 
out of state 

64% of the 103 who said they rarely find an interesting course to attend 
both in and out of state 



By merely adding the percentage points from among the three levels of 
evaluation/ i.e., often, occasionally, rarely, of Question 2, we find that for those 
who often find an interesting course to attend, the points total 121 , For those who 
o ccasiona 1 1 y find an interesting course to attend, the points total 147, ond for those 
who rarely find an interesting course to attend, 144, Thus it may be said that time 
seems to be considered a slightly greater hindrance to attending courses for these who 
occasionally find an interesting course to attend than for those who rarely find an 
interesting course to attend, and a substantially greater hindrance than for those who 
often find an interesting course to attend. 

By odding the percentage points for in state (38, 50, and 37), and the 
percentage points for out of state (39, 43, and 43), bo*h totals equal 125. So it seems 
that the physicians rate time [ust as much a deterrent to attending courses in state as 
out of state . 




The percentage of those who said loccJe was on inconvenience; 

! 3% of the 401 who said they often find an interesting course to attend 
in state 

17% of the 508~who said they often find on interesting course to attend 
out of state 

12% of the 17359^5" said they often find on interesting course to attend 
both in and out of state 



la 
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17% of the 460 who said they occasionally find an interesting course lo 
attend in state 

20% of the 274 who said th ey occasionally find an interesting course to 
attend ou t of state 

18% of the 684 who said they occasionally find an interesting course to 
attend both in and out of state 

24% of the 203 who said they rarely find an interesting course to attend 
in state 

28% of the 60 who said they rarely find on interesting course to attend 
out of state 

31% of the 103 wno said they rarely find an interesting course to offend 
both in and out of state 



By adding *he percentage points among the three levels of evaluation of 



Question 2, (see above) 



Often 13 + 17 + 12 =42 

Occasionally 17 + 20 + 18 =• 55 

“ 24 + 28 + 3 i — 83 



This suggests that locale is more a deterrent to attending courses as the Icve 
of satisfaction decreases. 

Also, by adding the percentage points from the three levels of evaluation 
(see above) for 

InState 13+ 17 f 24 = 54 

Out of Sta te 17 + 20 + 28 = 65 

the result that iocale is more a deterrent to attending courses out of stote than :n state 



Is not surprising . 



C. The percentage of those who said costs were an inconvenience; 



"1 often find an interesting course to attend. 1 ' 




Out of 


Both In o^d 


In State 


State 


Out of State 


Costs, such as 




1 3% 


u%~ 


Tui tion 


13 


ii 


i? 


Other Direct 


9 


14 


16 


Totals 


35% 


40% 


45% 
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"I occasionally find an interesting course Out of Both In and 

to attend. " [n State State Out of State 



Costs, such as 


16% 


16% 


19% 


Tui tion 


11 


12 


10 


Other Direct 


14 


14 


16 


Totals 


41% 


42% 


45% 


H l rarely find cn interesting course to attend. 11 


Costs, such as 


20% 


27% 


19% 


Tuition 


30 


• 11 


16 


Other Direct 


29 


45 


24 


Totals 


79% 


113% 


59% 



There is nothing unusual here, either. Costs are more an inconvenience to 
attending courses out of state than ir> state, and other direct costs are the primary cost 
factor. Costs were more meaningful to those who rarely find an interesting course to 
attend than for those in the other two categories and they listed many more other reasons 
why they seldom attend such courses ond programs than either of the other two groups -- 
164 for the rarely group, 98 for the often, and 91 for the occasionally group. 

One hundred three physicians rendered comments to this question, but they 
also added that they do attend. Their comments were added to the comments from 
Question 1 pertoining to reasons for not attending courses and to the comments in this 
section (3 d - ''Other reasons why you seldom attend such courses and programs.") 



Subject of Comments 


Number of 
Commen ts 


Residency 


136 


Not related to specialty or interest 


90 


Time loss from practice 


79 


Military commitments 


56 


Intern 


56 


New practice 


51 


No adequate patient load coverage 


48 


Family commitments and responsibilities 


36 


Incompatible with schedule 


35 
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Number of 

Subject of Comments Comments 

Solo practice 35 

Prefers other methods of continuing medical education 26 

Must attend too many specialty meetings 26 

Notin private practice (federal administrative, etc.) 25 

Type of practice (OBG, PHD, PSYCH, etc.) 18 

Retired or semi- re tired 16 

Advanced age 14 

1 1 1 hea Ith 1 3 

Fellow 9 

Courses repetitious / 

Inertia 6 

Courses too specialized 6 

Mediocrity of courses 5 

Church affiliation eliminates Saturday courses 4 

Last minute notification 4 

Too theoretical 3 

Lack of scientific content 3 

Courses not specialized enough 3 

Procrastination 3 

Lack of convenient parking 2 



QUESTION 4 

M*on)d you please made from 1 to I { 1 . JuM. \ L ast valuable ) tin* follow inu toni^-> 

.nnl programs ;u voiding; to tin ir o\ nail t iTet liwnoss in ynnr continuing imdieal < daication. 

/noepri f in* of i f rn r mourn ?mt/ r nU v. ( Plea'a 1 !ea\ e blank any which \ on do lie t ath ml. ) 

... ... .. One to T Imv Day S\ inpoMa ( I'nit i f.Mtv. Mo.iit Cam or Socieh . eh , 

Can. nit Canutes (l niu isitv. I ft.it t A^nc,. (am»i Siniitv. etc. t 

llci'nlar lltviaiiii’j Hospital Vio^aius iiziand toninb. <T(\ <. < iMMiltati o:iv 
... .. .. etc. ) 

( niaitv Medical Snciitv ( | f i » fc'r .m> at m< vtiiro • 

Animal Meeting \\ adiiii'j,(nn State M<i!iial Woi i, f f < : n 

Speviiitx (a mips, ( iin i I in^s ai id atn mal juojJrains , i 

.... _ KMrndw* Piotmadu.iV* (!oiiim s ( lw< » \nvb to ten w i a k** r -r inr 1 1 - in 

dotation in specific fiililO 

Otlei ( pic i\r **pccif\ ,'ndipadrl 



As mentioned earlier, many of those who answered Question 1 that they "don't 
attend courses often enough to make on evaluation" neveitheless did offer evaluations 
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of some of the items In this question, even though they were asked to leave blank ony 
which they had not attended. Shown below, first, are the gross results and, second, 
the results eliminating the evaluations of those who said they didn't attend often 
enough to make evaluations. 



Number 

Answered 


Grade 

i 1 J 2l 3 


l 4 


* 

6 ) 

-C 




Percent 




O 


2433 


45 


33 


13 


8 


1 


1427 


18 


32 


26 


21 


3 


2270 


29 


34 


20 


15 


2 


2035 


7 


16 


27 


47 


3 


1906 


10 


31 


32 


24 


3 


2595 


54 


28 


11 


6 


2 


1396 


43 


17 


10 


24 


6 


* 401 mentioned 11 


Other" 













Grade 




* 


Number 


1 


L 2 1 


L 3 i 


1 4 




Answered 




Percent 




6 


2092 


47 


33 


12 


7 


i 


1203 


19 


33 


26 


19 


3 


1877 


28 


34 


21 


16 


2 


1743 


7 


16 


28 


47 


3 


1669 


10 


31 


32 


24 


3 


2099 


54 


28 


10 


6 


2 


1130 


45 


17 


9 


23 


6 


* 286 mentioned 


"Other’ 


i 







One to Three Day Symposia 
Circuit Courses 

Reqular Recurring Hospital Programs 

County ^caical Society 

Annual meeting Wash. State Med.Asscc. 

Specialty Groups 

Extensive Postgraduate Courses 



One to Three Day Symposia 
Circuit Courses 

Regular Recurring Hospital Programs 

County Medical Society 

Annual Meeting Wosh. State Med, Assoc. 

Specialty Groups 

Extensive Postgraduate Courses 



In the final analysis, there is little difference in the two tables above, 

The specialty groups and the one to three day symposia, followed by 
extensive postgraduate courses, seem to be most popular. The county medical society 
meetings rated the lowest, but theic perhaps good reason why they do. They are not 
restricted to the purpose of assisting rn the continuing education process of the physician 
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and they draw from the entire spectrum of the physician population. The Washington 
State Medical Association 'Vine'll Meeting is likewise not solely for the purpose of 
adding to or shoring up the education of the physician and has many other facets, such 
as Association business, social, and associations in many other ways, including the 
opportunity for pbysicicns to talk to other physicians with whom they do not come in 
contact within their normal daily rounds. 



QUESTION 5 

C Wli.it oilier madia do yon like In use in “keeping up" v» ith vonr continuing im dicul education. 
P)«uvc giath 1 from 1 to -1 (1 host, \ least valuable) 

(Jcmral Journals' {JAMA. G.P.. etc. ) 

Tiirow-Aw a\ s (Medical A\ »rlcl News. \Iotl< ni Medicine. etc, *i 

Specialty Journals 

X fedical Tc\ ts 

. _ Audio Digest 

. Tel-K-Med 

Medii.il Kducatfnnal "IcleVtMcMi { sncli as Se.dllr (’.liafiiiel *) medic. 
programs 1 

Motion Pic tures 

Ollier (pkiKc specify ami grade) - - 

Question 5, an evaluation of other media in "keeping up", was graded os 



follows: 



Number 

Answered 


1 


Grade 

l_2_J_ _3_ 


1 « 


-X 

L. 

a) 




Percent 




o 


2994 


27 


35 


24 


11 


2 


2754 


11 


26 


30 


31 


2 


2958 


66 


15 


8 


8 


3 


2550 


25 


39 


22 


12 


2 


1288 


25 


19 


15 


35 


6 


815 


7 


9 


13 


58 


13 


1042 


10 


16 


15 


48 


11 


1263 


12 


24 


25 


35 


5 



General Journals 
Throw-aways 
Specialty Journals 
Medical Texts 
Audio Diges* 

Tel-E-Mcd 

Medical Educational Television 
Motion Pictures 



*A number larger than 4, a check mark, or other symbol 

ERJC is 

2'\ J 



There real!/ isn't much to say about the resu Its to this question . The numbers oF 
answers are relevant, as are the grades, and the percentage figures in the "Other" column 
are pertinent because there were about two to three percent or the physicians who used 
check marks or other symbols instead in assigning a numerical value from 7 to 4 as 
requested. Note that for the lost four items listed, the percentage points are higher than 
the 2 to 3 in othei cases. The differences are numbers assigned which were higher than 4 — 
numbers such as 5, 6, 7, 8, or 9. 

Comments (under "Other" [please specify and grade] ) with their numbers are 
shown below: 



Personal communication with informed colleagues 40 

Medical and surgical cli.Jcs 12 

Grand rounds 8 

Specialty workshops 7 

Detail men 7 

Seminars and conferences 8 

Monographs and reviews 5 

"New England Journal of Medicine" 5 

Local evening courses on various phases of medicine 4 

'Year BoA of Medicine" 3 

ACP self-testing program 3 

NME/TV tapes 3 

Closed circuit TV 3 

Library service for specific materials 3 

fn-Hospital meetings 2 

Symposia 2 

Audio tapes 2 

Exhibi is a t meetings 2 

Publications of leading pharmaceutical houses 2 

"Northwest Medicine" 2 

"Medical Economics" 2 



"Review of Surgery” 

"Hospi to! Practice” 
Subject-related books (not texts) 
Serving on hospital committees 
Correspondence courses 
"See Your Doctor” program 
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QUESTION 6 



6 



A IV llu k ;.nv ^nltjcft •ii'iuv \\lnV»i ymi Eivl h.i\c not rarivi-J iHicic lit ;ith ntimi !n rmitniiimw 
(’(Im utinM roiii'srs pro^r.rns? 

Vcs Q1 No □ 



tf "Vos * wnr\l vim please indicate tlnsr subject areas': 



This question, dealing with other subject areas for stud/, brought in the most 
comments. Thirty-two percent in the survey gave comments. Subjects falling within the 
broad classification of the practice of medicine were the most numerous, followed by 
subjects deoling with medicine and society, and basic medicine. Quite a substantial 
number thought that more courses should be aimed at general practice, Some specific 
subjects mentioned 'were allergy, cardiology, dermatology, What's new?, and practical 
courses for the average practice. Subjects falling in the general area of basic science 
were also mentioned in goodly numbers. 

The comments were too numerous to ’ st here. They will be classified into 
several groups and included in Phase II of this study. 

SPECIAL QUESTIO N 

.VH T: K 111 follow im; ipustmti is not u p ,nt of the (Jncvtimmairc pe r sc. hnl jtonr amwtr* to it i) I hr 
Vroiilh appreciated and most helpful. 



Wlut dn \on hrluvr me tli(' lurch fur i inplusis fifth u iiari! ft* juitu'tit mrr 

in y mr < nimnunih in the ni.m.iiii ivm n( <f lirjit ili^« me. tamer am? stinVv. I'tease rank hum 1 tn t 
( 1 .it i M. 1 : h avt lit i <1 >. Pi r ur li a\ e hi. ink ,tuy iUin with wlmli )nii . l » not familiar. 





RATING 


HEART 

! 

12 3; 


CANCER 

! 

1 2 3 4 j 


STROKE 
1 2 o 4 


Diagnostic facilities o/ 0 


(1363) ** 
18 18 19 44 


1 

(1427)** 1 

23 18 19 39 I 


1 (1304) ** 

19 17 19 43 


Rehabilitation l;i( illt'a 1 '* % 


" (1393) 

28 30 21 19 


(1256) " 
23 25 24 25 


(1660) 

56 24 9 8 


TV, tim’d ancillary pciMimic] o.^ 


(1349) 

31 30 20 17 


(1260) 

23 30 22 23 


(1486) 

46 34 13 10 


Physicians specially trained in: % 


(1230) 

17 17 20 44 


(1249) 

18 23 20 37 


(1230) 

22 23 19 34 




( 1296) ” 


(1318) 


(1312) 


Tieatmcnt facilities o/ 

/o 


18 22 23 36 


18 27 23 31 


27 28 21 23 


Kliiniiutiui of financial bniii«Ts 

to the availability ot Mr\ice so 


(1364) 

33 20 21 21 


(1454) 

40 22 16 17 


(1467) 

39 22 18 16 




(219) 

mentioned 

"Other' 1 


(220) 

mentioned 

"Other" 


(278) 

mentioned 

"Other" 



Note: Percentages will not necessarily equal 100%, because ratings other than those of 
1 - 4 are not shown . 

** Number of physicians evaluating question. 

This special question w as added to the original questionnaire and proves to 
be qjite interesting. It did not draw cs many answers os some of the other questions, but 
it was on the back where some may have overlooked it and, more important, these 
diseases ere not dealt with by all pnysicians. 

Cancer received the most evaluations in three of the seven items, and Stroke 
received the most evaluations in Four out of seven items. 

For the management of heart, cancer, and stroke, t Fie needs, as seen by t Fie 
physicians in their ov.n communities, were: (from top to bottom of the list are the greatest 
to the least need .) 

1. Elimination of financial borriers ta the availability of seivice. 

2. Rehabilitation facilities. 



3. Troined and Mary personnel. 

4. Treatment facilities. 

5, Physicians specially trained . 

6, Diagnostic faci I ities . 



For the management of these three diseases individually, the needs in the 
communities as seen by the physicians were: 



( 1 = greatest need, 6 = least need ) 



HEART 


CANCER 


STROKE 




i 


1 


1 


Elimination of financial barrl 


3 


3 


I 


Rehabilitation facilities 


2 


2 


2 


Trained ancillary personnel 


4 


4 


4 


Treatment facilities 


5 


5 


5 


Physicians specially troined 


5 


5 


6 


Diagnostic facilities 



There were a meaningful number of comments to this q 
are enumerated below. 

Facilities seem adequate 
Need for: 

Regional facilities centers 

Psychiatric rehabilitation 

Extended care facilities 

Rehabi fita tion facilities 

Coronary care units 

Occupational rehabilitation 

Expansion and upgrading of existing facilities 

Local medical control over needed facilities 

Consultation facilities 

Special care units 

Cobalt units 

Home services 
Convalescent centers 
Geriatric core 

Teaching of family members for home care 

More general physicians 
Physicians specially trained in: 

Neurosurgery 



*i J they 



87 



] 



? 

5 

3 

L 

1 

1 

1 



; 

2 

} 



20 



9 
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Neurology 8 

Radiology 3 

Surgery 3 

Internal Medicine 2 

Anesthesiology 2 

’’After Ho ts Clinic" operated on rofafing-physician basis 1 

Education of physicians in newer concepts of treatment 

ond technical competence 31 

Preventive measures and research 16 

Newer laboratory studies 3 

Studies on prophylactics 1 

Isotope scanning 1 

Lymph angiography 1 

Education of public 23 

Education of patient 11 

Better use of ancillary personnel 14 

Physicist i 

Speech there pi s ts 1 

Cohesive planning and organization 8 

Staff conferences with qualified outside personnel 2 

Quality control 2 

Inter-Agency (including MO) communication 1 

Adequcte retrievable records 1 

Elimination of vested interests which now prevent 

patient from getting available care ) 



COMMENTS PERTAINING TO THE SCHOOL OF MEDICINE, UNIVERSITY OF 
WASHINGTON 

Questions i, 3, 4, and 6 drew almost a hundred comments concerning the 
continu'ng medical educational efforts of the University of Washington's School of 
Medicine , 

In recent years, and more and more each year, the Unive r sity of Washington 
has been looked to by the medical profession to play a leading role in the continuing 
medical education of its members. The University has responded, even theurh St has 
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never received budget consideration from the State Legislature for this purpose. Its 
efforts in this area must, of necessity, be supported by tuition charged for the courses 
it offers- Because of land limi totions, greatly expanded staff and student body, the 
University also has had to build expensive parking facilities. These facilities must be 
paid for out of revenue generated by them. Ail who use must pay — students, staff, 
and visitors. Thus it was not unexpected that some comments were directed toward the 
cost af parking and fees for attending classes. 

To make this survey as useful as possible, all comments concerning the 
University were expected verbatim (but without Identification of the author) and given 
to the Dean of the School of Medicine. This at least will give him an opportunity to 
weigh some of the thoughts of whose who have attended his courses. Some of the 
comments were complimentary, some were not, but aJ! will be carefully considered. 
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appendixes 



Appendix A. 
Appendix B. 
Appendix C . 



The Clerical Operation and Budget 
The Questionnaire 
Paper Supplies 
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APPENDIX A 



T HE CLERICAL OPERATION AND BUDGET 

The purpose of this section is to present some of the clerical details of 
this survey so others planning to conduct their own surveys may consider what we have 
done. This is with the thought that it is well to profit by the mistakes of others, 
because one won't live long enough to make them all himself. 

The questionnaire (see Appendix B), including the letter of transmittal. 

Is all on one sheet of paper folded to four pages. This we thought important to prevent 
losing parts of the questionnaire and for ease of handling. The questionnaire was 
ordered from an outside printer to be delivered folded to fir in a Number iO size 
window envelope olong with a pre-addressed postage free envelope (Number 9). 

(See Appendix C for supplies.) First class mail was used. 

Three labels were printed for each address One was for the first mailing 
and the second served as a tickler file. As questionnaires were received back in the 
office, the corresponding second labels were removed from the file ond destroyed. 

The labels remaining in the second file were then used to address the second mailing. 

The third set of labels was used to mail the results of the questionnaire. 

The mailing address labels are the self-adhesive, size 3 1/2" x 15/16", 
ond are packaged 5,000 per box. We addressed our labels with a printer under command 
of on IBM 7040/7094 computer system. 

The addresses were obtained from the Medical Moiling Service, Inc., 

426 S. Clinton Street, Chicago, Illinois 60607. This is one of the nine or ten com- 
panies authorized by the American Medical Association to maintain and rent the master 



list of physicians . In our case we bought the list of Washington State physicians on a 
magnetic tope v/hich cost $250,00 and agreed to certain restrictions os to its use. 

The mailing of the questionnaires began < n March 25 and ended on May 3. 

At first the questionnaires came back rather quickly and in good numbers. By the time 
54% were returned, the second mailing began to all who had not returned the questionnaire 
from the first mailing. The questionnaires in the second mailing had a small 4 l/4 ,r x 
5 1/2" second notice attached by inserting it partly under the mailing label. 

Even as late as the last week of September questionnaires dribbled in, 
one at a time, but those received after July 18 have not been included in the final 
results. 

As the questionnaires were returned, the corresponding second labels 
were pulled from the tickler file. The questionnaires were then sorted into one of two 
groups. If they contained no remarks, change of address, or any other notes, they 
were placed In one group ready for key punching. If the questionnaires were returned 
with notes, remarks, or change or address, action v;as taken Immediately to update our 
files or to record the remarks or notes. From each pile, batches of questionnaires were 
counted and sent to key punch. When batches were returned, their count v/as verified 
to guard against loss. Of course, exact instructions were written for the key punch 
operators and a high degree of quality control imposed. 

The results of the first 689 completed questionnaires were processed 
through the computer to see that there were no programming faults. It may be of 
interest to the reader to note that the results from these first 689 were very close to 
die final results. 

At tins time the accounting is not complete, but the actuaf expenses are 
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known to be about $1,000.00 below the budget. Most of the savinys were realized in 
salaries. Also, it was found unnecessary to engage a consultant, thus saving $300.00. 
The final accounting for the survey will appear in t».e reporting of Phase 11. The 
budget for the project is on the following page. 
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3UDGET 



February 1, 1968 -May 31, 1968 



SALARIES 



1 Clerk, 4 months full time @ $375 .00 month $1,500.00 

1 Typist, 2 weeks full time @ $90.00 week 180.00 

1 Manuscript Editor, 2 weeks full time @ $1 16-00 

week 232.00 

1 Supervisor, 4 months 1/5 time @ $150.00 month 600.00 

1 Project Coordinator, 4 months 2/5 time ! §> $433.00 

month 1,732.00 

Total Salaries 



INDIRECT COSTS @ 2 1-5% of direct salaries 

EMPLOYEE BENEFITS - Retirement, Social Security, 

Medical Aid, and Health Insurance 
Premiums 



CO NSULTING SERVICE 

4 days@ $75.00 per day 
SUPPLIES & MATERIALS 



General (Office Supplies) 191,00 

Postage 818.00 

Total Supplies and Materials 



EQUIP MENT 

Computer tape with ^mes and addresses 
PUBLICATIONS 



Special Report (Survey results of 20 pages) 800.00 

Other (Questionnaire of 4 pages) 475.00 

Total Publications 



OTHER 

l! lush afions 



AMOUNT REQUESTED 




29 



2:u 



$4,244,00 

913.00 

509.00 

300.00 

1.009.00 

250.00 

1.275.00 

250. PC 
$8,750.00 



APPENDIX B 



'Ittaaljunitiur Atatp ifU'Jnral Animation 

HW-'t .7: .^r V> f h im IT u h I f it a r J 
^rattlc.AflHsIjmijtim i.i a ll 3 
ITthcliirlti j-yito 



THE QUESTIONNAIRE 



180359016 (Identification Number) 

GEO. ALVIN OJEMMO, M.D. 

1632 N. E. 12 AVE. 

SEATTLE, WASHINGTON 98105 



l)i\ir l’i nfcssional Colleague: 

I would like \ery much for you to complete flu* follow mg qiu stinrmairr and n tmu it to 
my office in (lie enclosed postage paid envelope. 

This queslinimaire lias been pre tested w illi fifty phvMc'ans in private practice selected 
at random from the Washington State Medical Association s membership files It was gem i.dlv fell 
tliat tin' questionnaire was uncomplicated, easy to answer, and inoffensive. I hope that son, too. will agm 
fm the inhumation we gain will he helpful in assessing future needs in the area of continuing medical 
education. 



Your name appears on the questionnaire. We w ill not sin 'je out amours' irplv , hut we do 
need the name so that we may use it in grouping replies by specialty of practice, him lung in pr.iclkc, 
age group, re gion, e tc. 

Results of the questionnaire will lie distributed to all physicians in the State ol Washington 
for their inhumation, so please help me to make the results meaningful by u pKiiig mhui. 
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STUDY OF CONTINUING ll)UC‘ TION 

ih (t joint effort of the 

WASHINGTON STATF MFIMCAK KOI CATION AND HFSKAIUII FOUNDATION 

and the 

WASHINGTON ALASKA ltFClONAF MKDICAL PIIOGHAM 
PIIASK I — Some Opinions of Physic unis on Cc liliiwing Kducution 



Will: re, Hard to cnntinnine medical • clnc-at i« h i chum s and other fnrin.il programs currently 
IninH made available In yon. re^ardh sv of their spomorshitK would yon nivc soar overall 
e valuation of the effectiveness ole om si. s and other formal programs by cl in A ins; one * d' the 
following statements: 



('heck One 



□ 1 am uswiUtf satisfied \x iM» lliosr I ha\r attended, 

j | h 1 am only \ouu-tivn's ; satisfied with those I have .lUundod. 
P~] e 1 don t of tend lliem often cnminh to nuke an e\ ablation. 



Comments: 



s-y \\ ith fi xat'd to the Mil»je<.t mailer oileird in pnHniadu.Je morses or oilier lurnal programs. 

" w mild you make an mural I t valuation h\ elir. kin" one c ■ I the follow inn >t a lenient**: 

il l n/V< n fi ml an inti lotini’ ennrsc to allend, 

P] In St.de p] Onlnf Slate ["] Ik. th in Slate and 



du ck mi n >> i (H<'(tsi(*iudhj line! an intne^liiu; inline to attend. 
Ajip/rmMe pj In Stall' pj Out et State 

J i I r<urlij find an inluesliiiH tourse to attend. 

[~] Instate p] Out of State 



Out of St.iti 



both in State and 
Out of State 



Ik ill in State and 
Out ot S( it*.* 



Check Chic 
or ytote 



^ Ken . i ril less of interest. 1 seldom alii i id si it h i oni se^ oj prnni a ins I leeauw r I imoi o i niei w . ■ due t 

P] a Tim.' 

Q !> I,m ile 

P J e ( '« »sfs, y k h as 
Pj 1 tuition 

PJ d i rt) u i din » t ins|s ( ti .: \ « I. 1 j« • 1 1 1 >11 % i ir nuod.it ini is s 

□ d Of! a v 1 1 .or mu u 1 1 \ \ou u Id. an ,Jf» n 1 ox |, t «*m o s and piovne as., 

l pl.MM spe« i f y \ 



o 

ERIC 



31 



Would von please Viatic from 1 to I ( 1 - host. i least valuable l tin 1 follow ing ohhm's 
and programs according to their mvral! effectiveness in your continuing m< dicnl education. 
orrs/Kc/nr i *f conn nit'iuc if ml costs. ( Please leave blank any wtiRli you tin r.ot atU nd. ) 

.. _ One to Three Da\ Symposia { University, lleai t Awic . C.iiuei Society, ole. 

Circuit Onuses (University, f frail Assoc.. (iniurr Society. t ‘x\) 

Regular P.ecnning Hospital Programs ( grand rounds. (TO consultation 

__ Ounily Medical Society (programs' at meetings) 

_... Annual Meeting Washington State Medical Association 

.. Specialty Groups (meetings and annual programs) 

Extensive Postgraduate Courses (two weeks to tin weeks or more in 

duration in spin fie fields) 

Other ( please specify and grade) 



Wliat oilier metlia do \<ui like to use in ‘keeping up*’ \\ itli youi continuing i ned real edm alien. 

Pie ase grade from 1 to -1 (1 - best, \ least valuable) 

General Journals (JAMA. O.P.. etc . ) 

Tluow- Awavs (Medical World News, Modem Mrtlieiiu-. i ted 

Specially [mu mils 

Medical Tex is 

Audio Digest 

TelK-Med 

Medical KduCiitioii.il T< lexeion < mi< h as Seattle Cb.mm 1 ‘I medic, a 
_ programs) 

Motion Pictures 

Other (please specify and grade) — 
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Vre llit ic any subject areas wlm li \on fet I h a\e i 1 1 > t ret i iveil Millieieht attt ntion in ionium rig 
t -duration courses and programs? 



Yes Q No □ 



If Y t "s \\ Oil Id you please indii ate tin no subjut areas: - . 





SPECIAL QUESTION 



NOTH: Hit* following rpicstinn is not a pari of the Questionnaire per sc. hut your answers to it will he 
greatly appreciated and most helpful. 



What tlo you believe arc the needs for increased empnasis it t7/j iriyrrd fo patient cure 
in your community in the management of heart disease, cancer and stroke. Please rank from 1 to 4 
( 1 . greatest, 1~~ least need). Please leave Mauls airy item w ith which you are not familiar. 



1 1 KAKT CANCKH STllOKK 



Diagnostic facilities j 

lh liahilitiition fac ilitie s | 

Trained ancillary personnel 
Musicians spec i<dly trained in: 

Tveatinent facilities j 

elimination of financial han k i s ^ 

to llie av.Jlahilitv of service 

‘ _ i 

' I 

Other (please specify and rank) ; 
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INDIVIDUAL CfWT.NTS O'! OULSTIGNS FROM 
REGION WIDE SYUVU.Y OK PHYSICIANS 



(usk: 

CQMMKNTS-on-STio:: 1 

Us u a 1 1 y satisfied 

Selective in courses attended: 
Application not practical: 

Too didactic: 

Not in private practice: 

Too specialized: 

Prefers circuit courses: 

Only s otre tines satisfied 

Courses not of practical value: 
Too c ler.cn t ary: 

Poorly prepared: 

Too general: 

Repetitious: 

Too esoteric: 

Poorly presented: 

Suggests small groups: 

Don f t attend often enough to rnkt 

Resident 

Nev practice or new* to area 
Specialty nec tings rostlv 
Intern 

Military service 

Prefer other nerns of c.n.e. 

Too special ized 
Fe 1 1 cfJ 
Retired 

Government employed 
Poor health 
No patient coverage 
Family corni t rents 
Hospital based FDs and con- 
sultants core to hospital 
to 1 c c t u rt* , etc,: 

Near re tire rent 
Fore convenient to attend in 
For 1 1 and 



Survey) 



8U 

15 

11 

7 

3 

1 



53 


Stereotyped : 


6 


20 


Too social: 


5 


10 


Intern or Resident: 


5 


7 


Teachers often 


3 


7 


pronoting selves: 





6 

6 

1 



eva ] u a 1 1 on f DLCAUSU : 

40 

29 

24 

21 

17 

7 

5 

4 

4 

3 

3 

2 

2 



2 

1 




Z:)'. 



i 
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coMMFNTS-ouhSTioh* 3 

Report}} css of interest, I seldom attend because of inconvenience due 
to: (Tire , Locale, Costs, were keypunched) 

0Tm:k 

I DO ATTKN’D (Most of these conrented on the inappropriate wording of 
the question): 103 



In residency 96 
Not related to specialty or interest 90 
fire loss f i or practice 79 
Not adequate pt. load coverage 1 46 
Military c on, ni treats 39 
Incompatible with schedule , 35 
Solo practice 34 
Family corritments and responsibilities 34 
Intern ’ 30 
Not in private practice (Federal, state, adm.) 22 
New practice 22 
Prefers other methods of c.m.e, 19 
Type of practice (Oli, Fed*, Psych,, etc,) 18 
Advanced iy-. ' * 13 
Poor health 10 
Courses repetitious 7 
Seri-retired k 6 
Retired 6 
Inertia 6 
Mediocrity of course 5 
Fell ow 5 
Church affiliation eliminates Sat, courses (SPA) 4 
Last minute notification ’ 4 
Too theoretical 3 
Lark of scientific content 3 
Procrastination 3 
Not esoteric enough 3 
Must attend too many specialty rcetinps 2 
Lack of convenient parking 2 
Too specialized 1 



cov v r.N rs ~on:fTio :; 4 

Would you please f.rade from 1 to 4 the fcllc/fnp courses and prorra: s 
according: to their overall effectiveness in vour c.r.c,, i) respect 1 vc 
of convenience and costs: (Keypunched) , 
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3 
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Short PC courses (3-4 days) 36 

Specialty courses at various teaching centers, in and 31 

out of state 

Specialty v'orkshops and clinics 13 

Clinical session AM A 7 

Snail group discussions with authorities in 6 

various fields 

Seminars 3 

Carefully structured preceptorships 3 

Conversations with colleagues 4 

Week. 1 v co nr^es at Ore en Ki ve r Cor.t.un 1 1 y Colic ge 4 

Closed circuit TV 4 

Specialty material on tape, slides, etc. 3 

Sub-specialty Journal Club 3 

Correspondence courses from various medical schools 2 

Overseas conferences 2 

Regularly recurring hospital programs 2 

Blue Cross 1 

Lederle Symposia \ 

Current literature 1 



COMMKSTS-QUKSTION 3 

What other media do you like to use in keeping up with your c.me.? 
OTHER 



Personal commun icatien with informed colleagues 40 
Medical and surgical clinics 12 
Grand Rounds 8 
Specialty workshops 7 
Uetnil ren 7 
Seminars 5 
Monographs and reviews 5 
•'New England Journal of Medicine*' 3 
Local evening courses on various phases of medicine 4 
Year Book of Medicine 3 
ACP self-testing program 3 
HUE /TV Tapes 3 
Closed circuit TV 3 
Conferences 3 
Library service for specific materials. 3 
In-ikv.pttal reelings 2 
Symposia 2 
Audio tapes 2 
Exhibits at roe tings 2 
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Publications of loading pharmaceutical houses 2 
"Northwest Medicine 11 2 
"Medical economics" 2 
"Review of Surgery 1 ' 1 
"Hospital Practice" 1 
Subject- re la ted books (not texts) 1 
Serving on hospital committees 1 
Correspondence courses 1 
"See Your Doctor" Program 1 



C OlPlhM TS-Or!'STT O a Jg 

. 4 . Subject areas which you feel have not received sufficient attention 
in c,m,c. courses and programs. 



Abortions and contraceptives, advances in 1 
Abortion laws 1 
Administrative medicine 1 
Adolescence 1 
Adoption problems 1 
Air pollution 1 
Alcoholism 9 
Allergy , 10 
Allergy for the GF A 
Anatomy 5 
Anemia, diagnosis before treatment 1 
Anesthesiology A 
Arterial angiography, techniques of 1 
Arthritis 6 



Audio tapes which can be checked out from medical libraries 1 
Automation in lab or at ory and clinical medicine, Impact of 2 



Cancer- „ 

Detection 

Management of inoperable cancer 

Cancer surgery, Paste principles of 

Kx review cour-cs * i 

Cnee i chcro therapy 

Breach open i c cu ref nor i 

Cancer of the rale genitalia and its incidence and 
discover; by average physician 



1 

31 

2 

1 

2 



2 

2 

1 

1 

2 

1 

1 



Basic clinical medicine and surgery 
Basic Sciences reviews and advancements 
"Backache" 

Biochemistry 

Biostatistics 
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1 
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Cardiology 10 

Cardiology for the CP 4 

Cardiovascular surgery, Surgical judgment in 1 

Chest 2 

Child development and learning 8 

Child guidance 2 

Childhood infectious diseases 2 

Civil defense 1 

Clinical sessions wi til patients 1 

Community services available for optimal 

patient care 1 

Connective tissue diseases 1 

Coronary care 4 

Counseling- 

Fami ly 1 

Marital 4 

Cytology 1 

Death investigation 1 

Dermatology 15 

Dermatology for the GF 11 

Diabetes 3 

Diagnostic medic e , 17 

Disability evaluation, HD's role in 1 

Disaster planning 1 

Drugs- 

Drug, therapy 14 

Use and misuse of tranquillizers, hormones, 

antibiotic, etc. v 4 

New drugs 2 

Immediate and long tern therapy o f pro- and 

pos t-adolcscent drug users (LSD, etc.) 1 

ECG interpretation 1 

Education, Need for stressing adequate education and 
avoidance of romantic and sexual entangleren ts 
in early life 1 

FKC courses, postgraduate 2 

Electronic equipment, use of advanced 3 

Emergency management ‘d care of patients 9 

Emotional aspects o " .'anted prognancoy, sexual 

delinquency, r 1 

Fro t Ion a 1 factors l sense 1 

LrphyscrM . 1 

Endocrinology 11 

KNT 9 

Environmental health 1 

Environmental red! cine as it pertains to 

specialties 1 
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Enzymol ogy 

Epilepsy 

Euthanasia 

Family practice 

Family planning 

Fees and relative values 

Fetal mortality 

Fluid-elcctrolytc balance 

Foot problems, diagnosis and treatment 

Forensic medicine 

Functional disturbances 

Functional illness vs* somatic illness 

Gastrointestinal diseases 
General Medical and Surgical updating 
General Practice-emphasis on developing 
more GPs 

General Practice 

Written tests for GPs ir. various specialties 
to point up need for remedial study 
Every area of medicine as it relates to 
General Practice 

''What's Nov" courses designed for General 
V rar ticc 

Practical courses for average practice 

C-U 

Geriat ri cs 
Group Practice 
Office gynecology 

Health education via mass media for patients 
Hematology 

Hospital pro cedi res, ”ew techniques 

Humanities 

Hypersens i t ivi ty 

Hype r tonsl on 

Hypertension, Surgical aspects of 
Hypnosis 

Iatrogenic disease 

Immunology 

Industrial redid no 

Infectious di?: cases 

Inhalation therapy 

In -U ranee 

Internal Fed i cine 

T n t r a - s ; e t: i n 3 1 y Interest s 
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i 

i 

i 

2 

4 

2 

1 

12 

1 

4 

1 

1 

2 

1 

1 

26 

1 

1 

1 

2 

1 

1 

3 

3 

1 

3 

2 

1 

1 

1 

1 

2 

3 

5 

1 

J 

1 

1 

8 

3 
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Laboratory procedures, new 

Laboratory tests and their diagnostic limitations 
Laboratory quality control program material 
Laboratory tests and their usefulness 
Lay participation in courses (behavior 
problems, etc*) 
legislation, Medical 

Malpractice 

Medical aspects of sports 

Medical care, delivery system of 

Medical comrrsunicatton (writing and speaking), 

The art of 
Medical economics 
Medical ethics 

Medical forms, Completion of, simplified and 
standardized 
Medical politics 
Medicare 

Medicine, changes in basic techniques of 

Medico-Ijegal instruction 

Mental health 

Mental health, preventive 

Mental retardation 

Microbiology 

Modern technology, Application of, to medicine 
Multiple injuries and shock, Management of 
Mycology 

Myocardial infarction, Management of 

Nephrology 

Neurology 

Neuropsychiatry 

Neuroses, Management of minor 

Newborn problems 

Nuclear medicine 

Nulri ticn 

Ob e s i t y 
OBG 

Occupational rcdicinc 
Of fi ca manor cr'cnt 

Office procedures, practical aspects of 

Ophthalmol ogy 

Orthopedics 

Orthopedics for the CP 

Otolaryngology 

Otology 
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1 

1 

1 

3 

1 

2 

3 

4 

3 

3 

44 

10 

1 

5 

1 

2 

20 

1 

2 

3 

1 

3 

1 

1 

2 

1 

14 

2 

1 

4 

3 

1 

4 

10 

1 

5 

16 

3 

11 

6 

7 

1 
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Pain , Rx 

Panel discussions of everyday problems 
Pap. smear, techniques of (by clinicians) 

Paramedical disciplines, Physician’s relationship to 
Paramedical education in all facets of medical 
profession 
Parasitology 
Pathology 
Pediatrics 
Pharmacology 
Physical fitness 

Physical medicine /Rehab t li tat ion 
Physic! an exchange programs 

Physician’s obligation to society (involvement 
in programs other than medicine) 

Physician participation in new procedures, 
techniques, etc. 

Physician-Patien t relationship 
Physician’s role in problems of youth and sex 
Physician’s role in society, Re-evaluation of, 
as health planner rather than health care 
dispenser 

Physiology - review 

reproductive 

Postmortems 

Prenatal care, Re-evaluation of traditional 
Pre-operative preparation and post-ope rative care 
Preventive aspects of degenerative dieeascs 
Preventive medicine 
Proctology 
P roct osigmoi d os copy 

Professional Courtesy: Docs it compromise 

patient care 

Psoriatic patients, Treatment for 
Psychiatry 

Psychiatry in General Office Practice 
Fsychophysiol ogy 
Psychosomatic Kediclne 
Pulmonary diseases and therapy 

Radiation therapy 
Radiology 
Renal diseases 
Respiratory diseases 
Rcsusci tat ion 

Screening equipment, Personalized instruction 
in use of 
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3 

2 

1 

1 

1 

1 

13 

7 

14 

3 

4 

2 

6 

4 

5 

1 

4 

20 

1 

1 

1 

3 

1 

10 

3 

1 

1 

1 

39 

21 

2 

4 

11 

1 

9 

1 

2 

1 
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Semantics 1 
Seminars 1 
Sexuality and human relations 3 
Social medicine 1 
Socio-Medical issues (drug abuse, sex education, 

VD, etc,) 16 
Specialty courses designed for specialist 

rather than CP 2 
Speech development 1 
Strokes 2 
Surgery- 

General 4 
Minot 2 
Surgical techniques, New 21 

Thyrotoxi cosis 2 
Toxicology 1 
Toxicology, Industrie] 1 
Transplantation 2 
Trauma 11 
Tumor therapy 1 

llrologv 2 

Vascular disease 1 
Vascular surgery 1 
Vehicle safety 2 
Viral diseases 3 

X-ray 2 



OTHEU COMMENTS 



Importance o c family physician on health 

care team, 1 

Make AMA more effective in helping practicing 

M.D. 1 

How to combat had publicity and sensationalism 

in lay Journals, 1 

Efficient use of medical care and manpower, 1 

l.iaison between Industrial and General 

medical practice, 1 

Hospital vs, home management of patients, 1 

Instruction in placing of an internal pacemaker 1 

Kelipion and Medicine, 3 

Home study sessions with follow-up at 
University, 
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Application of advancements in educational 
techniques (audio-visual aids, etc,) 

Greater fraternization among Nils. 

Correlation of specialties to general health. 

Actual technical demonstration and scrub-ins. 

M How to Unlearn" 

Through lord, or similar foundation grants, 

after ten years of practice physician could 
be placed back in original residency for 4 
to 6 months without having financial worries 
while he is being retrained in new techniques, etc. 
Joint practice and residency (evtrn 1/2 time, 

learn 1/2 time) 



C OMMKN T S - S P F. C 1 AT. QUEST I ON 
OTHER : 

Facilities seem adequate 
Need Regional Facilities centers 
Rehabilitation facilities 
Psychiatric rehabilitation 
Occupational rehabilitation 
Extended Care facilities 

Expansion and upgrading of existing facilities 

Local medical control over needed facilities 

Consultation facilities 

Coronary care units 

Special care units 

Cobalt units 

Convalescent centers 
Nursing homes 
Hone services 
Geriatric care 

Teaching of family members for home care 

Availability of house staff 
Need more general physicians 
Need physicians especially trained in: 
Neurology 8 
Neurosurgery 9 
Radiology 3 
Surgery 3 

Internal tVdicinc 2 
Anesthesiology 2 

"After Hours" Clinic operated on rotating- 
physician basis 
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l 

l 

2 

1 

1 

1 

87 

21 

9 

16 

3 

14 

2 

1 

1 

5 

1 

1 

7 

2 

11 

2 

1 

1 

29 



1 
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Education o I physicians t n never concepts of 

treatment and technical crrpctencc 31 

Preventive ^ensure r> and research 16 

N eve r 3 nb o r a t or y si u d i c s. 3 

Studies on prophylactics 1 

Isotope seeming 1 

Lymph a n g i o g r aph y 1 

F due alloy nt p nb lie 23 

Education ei patient 11 

Setter ?;m-' of ancillary personnel 14 

Keed ph>sitist 1 

Need speech the rapists 1 

Cohesive planning and organisation 8 

Staff conferences with qualified outside 

personnel 2 

Quality cone col 2 

In tor- Agency (including HD) communication 1 

Adequate retrievable records 1 

Elimination of vested Interests which now 

prevent patient from getting available care 1 
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THE CMK SHORT COURSE* 



by 
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The purpose of this study is threefold: 1) tc measure the reac- 

tion of participating physicians to the content and instruction of a 
short course; 2) to review the expressed reasons for their either 
attending or nol attending a short course; and 3) to see if, after 
part iexoat ircg in this short course*, any stated changes in practice 
occurred. 

The exact description of a short couse is not fixed, A report 
by the Bureau of Research and Planning Committee of the California 
Medical Association on continuing medical education indicates that "tv.'o* 
or three-day symposia are considered the most valuable modality 11 (6). 

In this study, a short course is defined as being five days or less in 
length, concentrated or continuous, and le^s than 40 hours in duration. 

The importance of information about the short course is seen by 
its extensive use in the United States and the lack of study of its 
effect on practice or knowledge changes, despite the fact that one sur- 
vey reports ’’most physicians acquire the medical inf ormation which they 
need by attending, , .seminars or by reading texts and journals' 1 (4), 
Approximately 47 per cent of the courses listed in the Journal of the 
American Mcdica 1 Association's annual review of continuing medical edu- 
cation in the Unitec States for 1968-69 meet the above definition of 
short course (5), While the short course, along with other continuing 
educational programs, has been increasing in use there has been little, 
investigation and some questioning of its efficacy# Williams ranks 
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short courses as least important among five basic forms of continuing 
education methods (6), Manning et al note that "In fact, there is 
little or no evidence that behavior is changed by attendance at conven- 
tional postgraduate courses 11 (3), Although this statement is not direc- 
ted specifically at the short course only, the method certainly is 
included , 

The most frequently encountered criteria for measuring effective- 
ness of postgraduate courses are attendance, gain in knowledge, value, 
or satisfaction. Very little is usually done about measuring the 
impact of the program on the change in practice of the physician vho 
attends. However, Ahrahamson notes that "In continuing medical educa- 
tion, our goals arc ul timate ly improvement of provision of health care 
in the community. In order to facilitate [this] goal, continuing medi- 
cal education has an intermediate objective [sic] changing the way (for 
the better) in which physicians practice medicine" (1), 

Through the cooperative efforts of the Division of Continuing 
Medical education and the Department of Urology at the University of 
Washington, a two-day continuing education program, "Urology for Mon- 
Urologists," was conducted on March 7 and 8, 1968, at the University, 

The course objective , as stated in the descriptive brochure, was "To 
present current thinking on some common urological problems and to ox- 
Pl ain the reason behind present attitudes." The program format was a 



rerics of short lectures (twenty to thirty minutes each), A moderator 
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for each half-day session node provisions for a cues t ion-nnd-nnswer 
period twice during each morning and each afternoon session, A total 
of 23 different presentations were made, 

SAMPLE 

The course attracted 65 physicians and 14 allied health personnel. 
This study presents data only on physicians. The participating physi- 
cians were represented evenly by those in group practice and those in 
solo practice. Sixty-nine per cent were in general practice, 9 per 
cent in internal medicine, 6 per cent in urology, and 16 per cenc in 
other specialties. Seventeen participating physicians were from states 
other than Washington, The average age of participants v/as 42, A fol- 
low-up questionnaire gave the average age of non-participants as 49. 

METHOD 

At the end of each session, the participants were asked to rate 
each individual speaker, using a four-point (Likert- type) scale from 
"exce llent" to "poor" with regard to 1) the relevance of the content, 

2) the speaker's ability to communicate his information, 3) his organi- 
zation of the material, 4) his visual aids, and 5) the participant’s 
interest in the topic# The participants were asked to indicate whether 
they thought the material was "new" or "review" for themselves# They 
were also asked whether they expected, as a result of participating in 
this session, to initiate any changes in their practices; "Yes," "No," 
or "Possibly," 
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At the beginning of the short course, ecic h participant was given 
an "inf ornaL ion sheet" to fill out indicating why he. attended. lie was 
asked to select one of five prescribed reasons as well as to respond to 
an open-ended question, "In my own wolds, my primary reason for attend- 
ing was . . , All participants were asked to estimate, through a 

four-point rating scale, the usefulness that each ascribed to current 
medical literature, specifically in the field of urology. Two weeks 
later, self-return post cards were sent to a random sample (n = 297) 
of those physicians who had been invited but who did not attend, 
requesting the information as was asked on the "information sheet" 
given to participating physicians concerning usefulness of current 
medical literature but also asking the physician’s opinion about whether 
he had kept "updated" regarding most recent advances related to his 
practice and what topic (s) would prompt him to attend a short course. 
Approximately nine months after the completion of the course, 
post cards were mailed to all physicians who had attended (n =* 61), 
requesting then to indicate 1) whether they had, in fact, filled out 
rating sheets while attending; 2) whether they had an opportunity to 
deal with patients with urological problems; 3) whether they had, in 
fact, made any specific changes in practice as a result of their attend- 
ance; and A) what specific changes, if any, they could pinpoint. Sixty- 
ono per cent of the physicians responded, BA per cent of whom had filled 
out ratings during the short course. 
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RESULTS 

Using means as scores f decrees of relationships were computed 
between responses on the various topics. Ratings of relevance and 
interest were highly correlated (.85), whereas relevance failed to 
relate significantly with ability to comrtinicate and organization of 
material . Abi 1 i ty to communion to and orgnni nation jof material arc also 
moderately correlated (.5b) with interest . Tims, interest is seen as 
one factor which cuts across all of the other variables rated. The 
only other significant relationship was between ability to c ommuni c ate 
and organization of material (.80). 

The most frequent rating for 17 of the instructors was a ’‘good” 
on four or more of the scales. Four instructors received modal ratings 
of “excellent” on three or more scales, and two instructors had scat- 
tered ratings below “good 11 on most scales. The average ratings did not 
decline with the passage of time during the day nor during successive 
days. 

As the number of physicians indicating that the content provided 
ideas for improving their practices increased, the number of physicians 
indicating that they felt that they would make a change in practice 
increased over the several sessions (r = .87). No other relationships 
were significant. 

Hie reasons for attending the conference arc given in Table 1. 

Tlie most frequent response, "Mot sufficiently updated,” was given by 
48 per cent of the participants. 



2/Z 



6 



Insert Table 1 here 



Of those attending the conference, 58 per cent viewed current 
medical literature as "of Jit tie use” as a resource for acquiring use- 
ful infornntion in the field of urology. Nonetheless, as Table 2 shows, 
33 per cent saw median; literature as “reasonably helpful” and 6 per 
cent viewed it as “extremely helpful." 



Insert Table 2 here 

Returns from non-participating physicians were received from 
53 per cent of the sample. Reasons they gave for not attending the 
conference are presented in Table 3, the most frequent being, "Could 
not free myself from patient obligation," followed by, “Program lacked 
relevance for me," 



Insert Table 3 here 



Of the non-participating physicians responding, 76 per cent 
felt that they were “adequately updated regarding most medical research 
relating to my practice"; only 7 per cent felt that they were not; 

17 per cent were not sure. This is in notable contrast to the replies 
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of those who attended , of whom only 48 per cent felt that they were 
sufficiently updated Cp< • 001). 

In four of the seven sessions, more than SO per cent of attendees 
indicated that they intended to malic changes as a result of material 
presented; for seven other sessions, over 50 per cent expected to make 
no changes. Percentages of change for the remaining sessions fell 
somewhere in the middle with responses in the category, "Possibly," 
accounting for the fact that neither "change" nor "no change" failed 
to reach the 50 per cent level. 

Of those who responded, 100 per cent had dealt with patients 
having urological problems; of these, 76 [jer cent indicated that they 
changed practice in soma manner or another and 74 per cent indicated 
specific changes. A summary of specific practice changes Is shown in 
Table 4, These arc grouped loosely into rough headings in order to 
cover the diversity of responses but, nevertheless, indicate the wide 
range of influence excited by the various lectures. 



Insert Table 4 here 



discussion 

‘Hint specific changes in practice are cited as having taken place, 
consequent to the presentation of a short course Is encouraging. Of 
the respondents, 74 per cent specifically indicated one or move changes 
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in their practices which they attributed to the vaiiour* lectures. This 
is not to say that citing chanf.cs is absolute proof of actual chance, 
nor is it to say that short courses are the most efficient method of 
teaching* But the implication is that changes in physicians 1 pivct’rrs 
occur as a result of information that the physicians gained during the 
short course* 

The reaction* of the attendee physiciums to the short course 1 
presentations and their content was favorable for the most part, 

Abrahams on has stated tnat there is some evidence that satisfarti i is 
related to degree of learning (1)* Fo this sample , it was apparent 
that positive reaction to presentation and content is highly related 
to relevance of interest for the lis ener. This information suggests 
the need for experimental research with the variables of knowledge gain, 
satisfaction, and relevance, using improved patient care as a dependent 
variable * 

The major reasons for not attending were whether the course had 
relevance and the fact that many physicians could not free themselves 
from their patient obligations. With 63 per cent of the physicians 
who responded citing these. tv;o reasons lor not attending, it would he 
veil for thos*. planning continuing nodical education courses in the 
future to reconsider the time when such courses are conducted, as well 
as to attempt to plan courses more relevant to physician needs. This 
information also agrees v?ith the finding of the California study 
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A point of interest is the level of agreement between those phy- 
sicians who indicated that they thought they would make changes in prac- 
tice as a result ol lectures and those physicians who actually were able* 
to cite .specific changes in practice nine months af teruurds* Percentage 
of changes indicated for all programs was compared with the percentage 
of changes for all responses. This percentage included only those 
people viic indicated that they would definitely change, and not those 
people who said possibly they would change* It was found that 31.6 per 
cent of the participants at the time 01 the lectures indicated "yes," 
that they expected to make a change in their practice. Of the returns, 
74 per cent cited specific changes, Even if all of those not responding 
failed to make any changes in practice, the percentage of change would 
still be 46 per cent for the total number of participants. Thus, it 
would appear for this study, at least, that expressed expected change 
in practice was a conservative estimate of reported change in practice. 

SUMMARY AND CONCLUSIONS 

In order to assess the change in practice effect of a short 
course, 1) a study of reaction of physicians to course presentation 
and content, 2) a study of the reasons for non-attendance , and 3) a 
measurement of postcoursc changes in practice were undertaken at the 
University of Washington School of Medicine through the Division of 
Continuing. Medical Education and the Department of Urology, A series 
of short lectures (twenty to thirty minutes each) with quest i on-and- 
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answer peri oris were presented over a two-day period. Participants were 
asked to rate each speaker on communication, organization, and personal 
interest n the topic* They v;ere asked to indicate whether, as a re- 
sult o r nrticipating in a session, they would make changes in their 
practices. Physicians who did not attend were asked to indicate their 
reasons for not attending. Several months after the course, participant 
wore querL.ru to see if any specific changes could he cited as having 
taken place in their respective practices* 

Results indicate that interest is highly correlated with rele- 
vance, ability to communion te , and organization. Most of the lecturers 
were seen as good or excellent. As programs were seen as more worth- 
while, thr amber of physicians indicating expected changes in practice 
increased over the several sessions. Those who attended apparently 
were less sure that they were sufficiently updated than those who did 
not attend. 

Short courses apparently provide knowledge that leads to stated 
changes in practice, although it is 'ealized that the amount and the 
importance of the change has not been evaluated in this study. The 
.lack of relevance of material and the physician's inability Vo free him- 
self from his practice seem to be the two nost important components for 
not attending this type of continuing medical education program. 
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tabu: i 



REASONS GIVEN BY PHYSICIANS TOR ATTENDING 
“OROJ.OGY I'OR NON-UROLOGISTS" COURSE 



N-33 



Reason 



Per 

Cent 



It Not sufficiently 
updated, 

2* Increasing number of 
patients with problems 
In this area* 



48 % 



13% 



3* Chance to learn and 
get away* 

i \ . To be familiar with 
latest research. 

5. Other. 



15 % 

15% 

9% 




TABLE 2 



COURSE- AITENDING PHYSICIANS * RATINES OK CURRENT 
MEDICAL LI XL PAT U RE AS A RESOURCE FOR CULLING CUT 
USEFUL IN FORMAT! ON IN UROLOGY 



Useful Index 


Per 

Cent 


1 . 


Extremely hilpfu] (most 
important source I use to 
"update 11 say practice). 


62 


JL * 


Reasonably helpful (I often 
use this source), 


33% 


3. 


Of little use to me 
(occasionally helpful). 


58% 


4. 


Of no use to me (an 
impossible task)* 


— 


5 . 


No comment • 


3% 




O 
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tabu: 3 



REASONS GIVEN BY PHYSICIANS FOR NOT ATTENDING 
"UROLOGY FOR NON-UROLOGISTS ' COURSE 





i 


Reason 


Per 

Cent 


1 


Could not free myself from 
patient obligation. 


33% 


i 


Program lacked relevance 
for me. 


30% 


i 


Unsati of actory previous 
experience, with short 
course. 


1% 


I *• 


Inconvenient location. 


;% 


■ 


Personal or family reasons. 


12% 


6. 


Too expensive. 


5% 


1 7. 

i 


Ccher. 


12% 



I 

I 

1 

! 

! 



I o 

me 



2* 



u 



TABLE 4 



AREA OK SPECIFIC CHANCES IN PRACTICE 
ATTRIBUTABLE TO THIS COURSE 



Area of Change 


Number 


Gonorrhea 


8 


»x 


8 




8 


Better therapeutic 
approach 


3 


Catheterization 


3 


Mis cellaneous 


3 
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Each year, the University of Washington School of Medicine liar tra- 
ditionally offered a series of two-day courses designed for the continuing 
education of practicing physicians. Historically, there has beer little 
attempt to assess the' outcomes of these courses. In recent years, however, 
both the number of programs offered and the attendance at each program have 
increased sharply. With the' increase has come a demand for formal evalua- 
tion. 



gain in knowledge of subject-matter content by administering pre- and post- 
tests. This system has four major disacvmt^ges : 



0) It is an arduous and time-consuming task for the test con- 
structor, the administrator, and the examinees. 

(2) Emphasis is placed on finite factual material. There is 
ample experimental evidence that this kind of material is 
quickly forgotten. 

(3) Emphasis i~> placed on product rather than process. 

* 

(^) The use of knowledge of content as the criterion implies 

a judgement of the speaker's performance, and the participants 1 
competency rather than the quality of the program per sr. 



One approach to the evaluation of short courses has been to assess 
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The authors believe content testa to bo inappropriate for in- 
service courses. A great dc il of research, over the years, has demonstrated 
the ephercov'jl nature of factual knowledge and the relative permanency of 
learning sots, i.e., study habits ar.d skills, the attitudes towards and 
motivation for learning. In addition, it is obviously impractical to at- 
tempt a comprehensive treatment of a complex subject in one or two days. 

It follow* , then, that dissemination of knowledge per so is an impractical 
objective for short courses such as these. Rather, we should be concerned 
with stimulating the search for new ideas and encouraging the full use of 
present knowledge, Consequently, sn instrument which is sensitive to a 
participant's affective response t* a session, a measure of his enthusiasm 
or motivation, should be a valid predictor of the extent to which the session 
will influence his professional development. 

A second approach to the evaluation of in-service courses has beer, 
to ask participants to rate the sessions on a number of qualitative dimensions 
Although this approach is free from the judgemental overtones of knowledge 
tests, the usual questionnaire does not permit the ratings to be summ.cd over 
items. However, the concentration on motivation provides a rational unity 
which permits the questionnaire items to be treated as a unidimensional 
scale. Thus, the effects of errors and misinterpretations can be ameliorated 
by averaging over related item 



THE IHSThblKIJT 

The instrument described in this study was developed along the 
lines of Osgood’s (3957 ) semantic differential. 1 Sets of bi-pol nr adjective; 
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C. Osgood, G , Guci , an \ ?. farmenViun, Tju »_ !Vy ■ r. uv c r i " n t of Meaning, Urbana, 
Illinois: University of Illinois Reece , V) l Jl . 
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- " were* constructed to asses;: genera] value (goodness) and relevance. In 
order to relate the judgments directly to the program, os opposed to In 
lecturers and r rgrr.i ^ers , two separate groups of items were developed. 
First, pari : eipar.ts './ere- a si; el to choose the adjectival values which test 
described Uiv objectives or aims of the session as set forth in the pro- 
spectus. becond, th<n r were asked to choose the values which best de- 
scribed 0]\ content pr'sen^d during the session. In the content section, 
two kinds of relevancy were, assessed: (l) the relev ncy of the content 

to the needs of the participants, and (2) the relevancy of the content 
to the lecturer's stated objectives. In addition, an item assessing the 
familiarity or novelty o c the content was included. The average of scores 
for items in part one was called the objective score (o), the average of 
item sco} as in part two was called the content score (c) and the overall 
average was called the motivation score {l \) . The completed motivation 
scale, with random rearrangement of iten poles, appears in Figure 1. Hie 
marginal parenthetical comments illustrate the plan of the instrument and 
did not appeal' on the printed version. The accompanying que sti on nai. re- 
appears in Figure 2- 



The Motivation Scale was first used at the eleventh annual meeting 
of the Washington Academy of General Practitioners at Olympia, Washington, 
in Juno of 196&* A useful instrument must permit statement regarding the 



(insert Figures 1 and 2) 



VALILATIOil PROCEDURE 





relative merit of sessions. CcnseipuenOy, iir !! -ti vat ion fcu-ln wn ; afrin 
ictercd a-:, a criterion variable in a simple errerU .f. In order V) pro- 
vi l'j ri _"it; .ate of -onci .ron 4 validity, a mu * ‘ . r.mnaire vns ruimirie teivd 
which iOl.i.-d each participating jhyc.ici.au to o'li'iat.v the number of patien 
he saw each year with problems related to too tjpic die cuc.ee cl, and to etc 
whethe 2 * ho c ;cpe c to d t o improve h i s <i i a g: - on t 1 • s t m: a t me at , pat i o n t ir.an r: go - 
ment, or referral procedures as a result of the a l r s 1 ■: n . Reopondenfn wer 
also encouraged to make evaluative comments myirliig soy aspect of + 1' e 
program v'h ! ch seemed noteworthy. 



Design . The experimental design in coir 1 tvo lecturers labeled 
Dr. A and Dr, B. Each lecturer gave two presentations of the borne mate- 
rial. In or.e presentation, a lecture method (L) v/as used; in the other, 
^ve ' 'Ojis or problems were pirescnted so as to require an overt response 
from each of the participants. The latter was labeled the involved ir.etho 
f ) . The design is suirunari z.ed schematically in Figure 3- 



(insert Figure 3) 
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i. . Re Ziab i lily . The c orrol at i on matrix for the combine d g vo\i\ .s 

is reported in Table I. The obtained correlation of each of 
the four objective iter . 3 with the total objective score (sun 
of item" one to four) ranges from , 80 to . 85 * The average of 
these correlations, calculated via Fisher 1 s Z transform , was 
taken as the reliability of the objective scores (.82), 

Item number eleven (new versus old material) yeilded consider- 
ably lower correlations than any of the other content items. 
Since the perceived novelty of the situation was a dimension 
of interest, the item was retained in the total scale. How- 
ever, only items six to ten were used to form the content sub- 
scale . 

Tne correlation of the content items (six to ten) with the con 
tent score (sum items six to ten) ranged from .72 to , 87 . 
The average of these correlations ( . 00 ) was taken as the reli- 
ability of the content scores. The average of the correlation 
of items one through eleven with the total score (sum of items 
one to eleven) was taken as the reliability of the total score 

(r =■ . 69 ) . 

2. Validity . It was an assumption implicit in the motivation 

scale that th_ objective '\nd c .ntent of a session could be s^p 
arately assessed. Evidence 01 the validity of this assumption 
appears in Table I. 




(Insert TaMc l) 
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The objective total had an average correlation of .82 vi th 
the objective items, but on average of only . h6 v:ith the con- 
tent items. Similarly, the content sub-total had an average 
correlation of .80 with content it '-ns, but only .1*7 with ob- 
jective items. Both these differences were veiy significant 

(p < 001). 

The primary evidence of the validity of the scales was pro- 
vided by the correlation of each item and sub-total with 
’’outside" indices of \,he relevance of a particular session to 
the participants. It was considered an indication that a 
session was relevant 'fa participant reported that he ex- 
pected the presentation to; 



(1) 


improve his diagnoses. 




(2) 


modify his referral de:isions. 




(3) 


improve his treatments . 




(U) 


improve his patient management. 




(5) 


increase his confidence i.i his 
practices . 


present 


(6) 


relate directly to problems he 
practice. 


neb in his 


(T) 


be of no benefit to his practice. 



For each of the seven variables score, the correlation with 
each item score, the objective score, the content score, and 
the total score was calculated. In all variables but six, the 
participant simply indicated whether the expectation vac pre- 
sent or absent. In variable six, participants were asked to 
indicate for an "average" year the number of patients they 
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would expect to lave problems directly related to the presen- 
. tat ion. 

A significant positive corrclrv * on was considered indicative 
of validity in all but the "no benefit” response, where a sig- 
nificant negative correlation indicated validity. It was ob- 
served that all but one of the questionnaire responses yielded 
significant correlations with every item on the motivation 
scale; the "modify referral procedures" response was signifi- 
cantly correlated with onJy five of the eleven motivation 
items. When questionnaire responses were suiaraed to form a 
single score, the correlation with the total scoie on the moti- 
f vation scale was ,50. This was interpreted as the overall va- 
lidity of the s:ale. 



(insert Table II) 
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The mean scores and standard deviations of the objective and 
content scales are presented in Table III. It was observed 
that Dr, A ! s ratings were considerably higher than Dr. B ! s 
on each scale for both presentations. The differences between 
content and objective scores for Dr. A were almost identical 
from presentation to presentation. The difference between 
objective and content scores for Dr. B, however, was signifi- 
cant in each presentation, and was Considerably larger in the 
involved presentation than in the ncn-involved presentation. 
Since this agreed with the written eonncilo of participants. 
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it vr.r. concluded that the objectives and content were rated 
differentially, arid that the scores reflected actual differ- 
ences in quality. 

(Insert Table III) 



O 
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Particular import?/ ve was attached to the difference "between 
the mean for objective items and the mean for content items, 
since this vas an index of the extent to which ohe objectives 
were accomplished. The written comments of parti cipents vas 
solicited on the accompanying questionnaire (Figure 2). Com- 
ments fi'om both Dr. A*s sessions consistently used adjectives 
such as "interesting", "challenging", and "scholarly", ho 
hostile comments were ire.de. For Dr. B the comments were quite 
different. In the lecture session most people made no comment, 
Orly three people called the session"interesting" , two con- 
sidered it "valuable", but six labelled the session as "bor- 
ing", "too text-bookish", or "dull and uninspiring". In the in- 
volved session, however, literally everyone common ted. Only two 
people cabled the session "interesting", one called it "chal- 
lenging". The remainder were derogatory ; they used comments 
such as "bigoted", "a hostile lecturer", and "argumentive" , 

From the written comments, it appeared that the involved and 
lecturer techniques were roughly equivalent for A; B, on the 
other hand produced observable negative reactions with the in- 
volved technique, but only apathy with the lecture. Each of 

2\‘0 
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these observations was reflected in the data- it was of par- 
ticular interest that the (0-C) scores for Drs. A end B were 
significantly different (p < .01) . The three items directly 
concerned with the relationship between objectives and content 
are ( 7 ) content related to objective, and (8) content fulfills 
objective. The mean ratings for these items are reported in 
Table IV. It was obsei'ved that all differences were highly 
significant (p < .01). Since the differences agreed with the 
writ tc-ii comments it was concluded that the participants were 
in fact rating the sessiojis in terms of the extent to which 
they net their stated objectives. 

(insert Table IV ) 



SUMMARY AND CONCLUSIONS 

1. Participants of this study rated the stated objectives and the 
contents of the presentations differentially. There was, how- 
ever, a pervasive evaluative factor: the ratings on the objec- 
tive scale were affected by the quality of the presentation. 

Th is difficulty could be overcome by administering the section 
of the rating scale dealing with objectives prior to the pre- 
sentation . 

2. The findings of this study indicate that the difference between 
means for objective and content items provides a valid assess- 
ment of the participants' perceptions of the extent to which 
the lecturer succeeded in meeting his own objectives. 

O 
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3. The findings of this study support the validity of the Motiva- 
tion Gcale for comparing sessions vith regard to the perce ived 
relevance of the objectives and contents. It is expected that 
continxicd experience vith the scale would provide norms which 
would permit meaningful statements about the overall quality of 
a session to be made. 

Limitations 

Evidence regarding the extent to which th a 1-lot ivation Scale predicts 
actual changes in a participant * s subsequent behavior was not provided. 

This is a necessary but time-consuming task, to which we are presently ad- 
dressing ourselves. In the meantime, the scale may be taken as a valid 
measure of intent, and thus useful per Sc . In the absence of instruments 
which have been field- validated, it may also serve as an estimate of the 
effect a particular session will have on participants 1 subsequent behavior. 
It should prove particularly useful where short administration times (one 
or two minutes) and non- judgmental overtones are desired. 
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igure 1. The Motivation Scale 



2:00 - 
2:30 - 



Dr. B's 
Dr. A's 



Figure 



O 




Room I, Group I (n = 96) 

2:30 Dr. A r s lecture: no participation 

3:00 Dr. B T c lecture: participation 

Room II, Group II (n = 55) 

lecture: no participation 

lecture: participation 



3, Schematic Representation of the Design 
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Figure 2- Accompanying Questionnaire 
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7 he Consultant's Letter 



Progirss 



H A 1 G H FOX. S.A. ' WILLIAM 0. POBER1SON, M.O., Sc--i-ilc\ \ 



l.cltir . y in referring pfitj.'M'ians, niititn frj rori-SM/k.'iifv oil th c 
staff of the I'niiersitrj lh<s\*ituh Seattle, were studied for *e> Urines* 
of the information transmitted. Scientific content fv cou\iden d 
(tdeiftu ite hut the letters lacked ctniecni for the patient as an 'ndi- 
vidunl in his <jrt 11 em ironnaut. The authors deplore f!\(‘ fact that 
this opportunity for (inching r< seldom grasped htj the cinotdluui s. 



A s stressed two years ago by 
one of us, tin* potential val- 
ue of tlie consultants report as 
a means of fostiring continuing 
molical education bas been nei- 
ther fu!l\* appreciated nor ex- 
ploited.* To furtliiT substanti- 
ate tins claim and estimate the 
utility gap, w e imdcitonk tin* 
following analysis. 

method 

We obtained copies of b’(i 
consecutive consultant s' lclleis. 
typed over a two-day pencil by 
tlie secretarial pool at tlie Dili* 
vi'.ity Jlospil.il, Seattle. Twentv 
wa re discarded became tic y 
wire either written to Uon-pio 
fessinual iccipiinls r»r simply ic- 
prmlucid form letters inti ink d 
to accompany hospital din-huge 
suinui. nit s. ( No dische* ge sum- 

5.ip|*'ir'ol ili :... 1 L I v I’nMu' H< i*h 
SciMte M.T.n.n: 1’1I I'i-'.vlV, .1,-1 :i> 
| nl lo | r- t l in .■ -I fii-:n *1 l.< N.'V-nvi’ 
Fmi l fnr Mi ’u.il I'rlm.ilLon 



niaiics per ac were included.) 
Tbe remaining S6 letters to re- 
f erring physicians uvie re .id and 
ipialilativdy scored uccmding to 
tlie following criteria: 

]. Terhnind Content, Did the 
report summarize tlie patient’s 
presenting problem and describe 
in general terms wlut was done? 
Was a differential or precise di- 
agnosis included? Was a thera- 
peutic plan outlined? Were 
specific*', ef follow-up included? 

2. 7/e/b/rV Comma Did the 
ceiivdfaut indicate nr an tj tnitj 
an ..cf|iuiut.iri, e w itb, or c m 
din for. tbe "total pa1ient"--i.e., 
was tlie evaluation and treat- 
ment kiih'.ed for tlie patient in 
iii'. own ciiviroiirm nl rat! ■ j tlun 
just for bis disease? 

3. Style and Tone. Did the 
foi ilia! r if the let U 1‘ <1 i< ut tin 
le.idei rapidly and Was its tmie 
positive ami di-mt? Was pionf 
leading adi ipuie? 

t. fh fmnfiotml Spuife s. Did 
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TABLE ! 

Infcrmalioncif Confer, t of £6 Co-oulto! ■! Reports 



Pt'lYl itt 



( o nit ni C utc goaj 


lTcsnif 


A. Heiter.itinn i>f p.i'ienl pruhlci.i 


70 


lh l)eM ription of clinic or hospital activities . 
f ! 1 >cfinitive diagnosis 


SS 


1 SpeciEe diayaiosi* 


G7 


2 \Vorking impression . 


94 


]), 'rherapeutic plan 


- 91 


K. foil nv-iip specific s . .. 


SI 



tlie consultant make any pei- 
ceptible cflort to oiiPnlmte to 
the continuing learning of the 
referring physic :.u; . J Were any 
refi’K JKVS : l'K'llului!y 



lesLfifi 

Ax noted in Table 1, tlie tcch- 
nieil content— in fix o specific ca- 
tegories was remarkably com- 

plete. And u Jit n a letter wan 
fanl ted, gencrallx only om of 
the points was emitted. Approxi- 
)v ito.v out* in fmr letters omit- 
ted a single hit of inlornutinn 
judged hy ns to lu* critical. ei- 
ther for complete comprehen- 
sion or total management of the 
patients problem. These were 
items such as drug dosage, re- 
turn visits, or equally peitinenl 
matters. Direct or implied state- 
ments re fleeting llie ; c ui % it I f .tu fVx 
aipu'nl mce with, or enmem 
for. ti e w 1 h ill pjtient were de- 
li e table in only AO peie i n( of the 
i rp-.ii l*-. TO pci e i ni made no iiT- 
ereiae \> hatpin \ « i tn this item 
Snmi u!m( in cuiiti.M. in ,SS p< ? 

t i n! of the h th is the M\ !e ,ilid 
I'M ie w i i e jndgi d ,;s "din a I. p- is 
itixr. and bawl/ I’.iJily-MX pci - 
c « nt had no major lyp igi aphie d. 
trail*-- i i pi ion al, nr dietafimnl < a - 
CIS. 

( )lll\ .'iil pell i rll it l! in ll p *, In 

o 
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prevented evidence that the con- 
sultant had seiod die opportu- 
nity cilliei In convey to die ie- 
feinng physician U'lexanl new 
infornialion or to reiterate, re- 
inforie, reevaluate or remrgani/r 
existent in. nrmation. 1‘urlher- 
moie, in not a single instance 
teas a ref n't nee to either a text- 
book nr journal article inclttdt'tV. 

discussion 

To a large extent, this analy- 
sis of infoi .nation content em- 
n 'borates referring physicians' 
pu captions of mic?i content, as 
reported in lOofv In )>oth in- 
stances the letteis were seen ns 
being considerably more com- 
plete than in the .study hy de 
Al.iicon, it ah and their review 
of this topic in Ihitain.- ((inn- 
ew'vabh this ihsp.nify in evm- 
pl i 1 1 • • s s i*. limply .i Kill'd ion 
of the use of tin dictaphone and 
tvpiwiiVr in the Ihiited States 
.is opposed {,} the qu il in "llH'l- 
i\ olde Knglande. ) Mon ovri . 
lh< "l.uk.idai- il attihite* de- 
licti d b\ (T miberlin' was iio- 
uln re di lilted in the letters 
studied. 

A specific and uIiAanl mnis- 
s;nii was leiv ma n d in one of 
t \ VlV h '1. 1 iep- n!s. ) *u: i mph.i 
sis. this is i qiuht.ilix r ,issi s s 
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incut, obviously depondml upon 
a judgment fact or in the evalu- 
ators. If we erred, it wax be- 
cause the study was purposeful- 
ly directed at overestimates of 
completeness, Conteix ably. still 
inure data need be included in 
consult ails loiters. Possibly pm- 
\iding \ printed check sheet of 
critical components of such re- 
ports to tin* dictating physician 
at the lime same time lie proof- 
reads and sign In’s letter won H 
help avoid omissions. This al- 
ternative is to he explored. 

Thai acquaim.mcc with, or 
col k< rn for, the total patient wws 
demonstrated in less than one 
lim'd of the reports is perhaps 
simply a reflection of the spe- 
cialized rob plaxetl hy the c<m- 
Miltant. Indeed it might well he 
argued that vich is tin* apprn- 
pii.de lole foi the technical >pr- 
i i ilM w lh tl it inti gr.itix c fuac- 
lion to hr pci formed hv fhe i e- 
feiiing gemT,tli>!. Alw.ixs tlie 
«pn stion must he asked, lmw- 
cxer, “Can definith i- Ire. dim nt 
plans ;ic tually lie di-M Tip. d 
xx it I ii n 1 1 being tempi red lw (on 
xid< rations of tliis snity"’ 

IT .ally, the lilajiil' pm pose nt 
lids .-u.ilyxis wax to .ipj i.i.M tin 
n 'iisull.i,il\ ] t ptf) [ ha its con 

tinning < duration pultnli d and 



utility. Fr ,.sd.I\ . 1 J u 1 ) emits In n 

\w rr dis.ippr ITihji.:: 1 1 . ! 1 timid 

: 1 1 hand is nnt 1 n ■ 1 1 1 yi used. \o 
<l.i t .( nfr |)’T.s< nlly l\ Lijl.il »!< ' do:> 
i i n i f a 1 1 i . ■ "j; either l ! h ■ refcri'int: 
y > / c. i , m "s thuii'diK in ibis mat- 
ter n r ll'it 1 elh-ctiv rn‘‘SN nf such 
c-fi'dus asMoniw; they me « n 
,is important and .issumiiej: they 
are undtrlalrii. J5.li. its null'd in 
{lie lbfin report. ;i:id a> support- 
ed in learning tint try, tin* po- 
lejtli;il of such letters should be 
significant. dealing as they do 
with highly relevant situations, 
in ovcuoirmn; die physician's 
‘ newdnfonualion 1>;i rricr** or in 
motivating immediate behavior 
eh mgr, C'npj tali zing on this po- 
tential. either by calling alien* 
lion to, or providing avenues 
for, firlher oxpluraiirm of role 
vnnt points, obviously lias not 
been done in tins s imple. To ini- 
tiate s leli >(fntls we should Id.e 



ti » propo-w l! it i vpm’/tV, ;-</<• 
amt, and "loily ret- 

crcnee Iv ended to every 
neb Viler n: j epnrt in lb ■ In- 
line- The implementation of the 
HrgioiiJ Mttl/ea! bibr an ole 
male.. diflicc.Jly wit’ this Iasi re- 
turn nu nt. \c>: only would Mic h 
refer* ncr permit the reader an 
avenr.e fov follow-up -should lie 
rhoo-se to explore it (and a Id 
percent exploration lalo could 
be considered successful )--bnt 
also the reference by its cry 
presence, would remind both 
writer ami reader of the cch na- 
tional potentials of ibis pavlitu- 
lai vehicle. If lefci cutes are r.nt 
avail ible at the consultant's fm- 
gcilips. or in his card fiV, be 
ea.j easily cull them from ' Cur- 
rent Medical liefer dices ’* or the 
appropriate ' Vearbunb, 6 thus 
pi(in»olii«;4 bis u * n conlimimg 
!(vi; ( ii:<y 



sur rnory 

luil Ion jug up a pro in- is study 
of llie eojisiillarit's rrp< i t aid 
liiiw it I*- viewed by the rc-viv- 
er. w e have analyzed th • infor- 
mational content nf a sample of 
Sfi such repoils -pai [iiularly to 
tlete* inii.e then conFiniing edu- 
cation ingredients, While tin 
tuiilcuts were judged as re!a 
tiuny complete in le.ms of tlvir 
primary purpose (om in four 
still had signific aid emissions), 
less than 30 percent conveyed 
any glimmer of continuing edu- 
cational light. \pprnding lml a 
single reference to each such let- 
ter would not only service tin* 
reader but would aim focus the 
writer’? attention on the * ■>. Mu- 
lling medical educational poll n* 
tial of this medium. u 

tbmei sihj o j W'm/u ug/ui: 
School 4 >J Mt'tfh inr ;'b S' / ') j ) 
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I ,i si J i . !,r saw id i i : i > i 3,1.11 k’ (pobu!i;;~-JiAfCbAef— 
J become ;iv.i f Table fur general me. Ad- 
min iMri rd t:i llif pu \'i' mdy unsi nsl i.e-u :u 
ueg live ]i'(nl] M-; w illun the firs-r ih-72 hours after 
her dciiveiy of an Rh positive offspring. or fol- 
lowing an aboitir: - ! or mist awi.ige, RnoCAM is 
aimed at preventing Ik r snisi!i/,itio;i and cl? com* 
venting Rh pvi ■! d n:ss with w subsequent preg- 
n.umy. The responsible mc'jhanism apparently 
involves the immune ghmubn’s afhnilv for any 
Rh positive fetal cell:, which mas have escaped 
into the nvileir A eh dilation at the tunc of de- 
liver/ and which, left to theinsclves, v.voiu prime 
jin for ii.i i trinity trouble. The globulin-red cell 
combination seems somehow to eliminate the 
antigenic poleiilnl rrf the fetal red cell. Thus, a 
simple injection has the potential to virtually 
eradicate hemolytic disease of the newborn due 
to llli incompatibility and to push its ultimate 
effect (a pel m meudy damaged infant) and its 
current treatment {exchange transfusion) far 
down Inc road toward extinction. 

In many ways lias very mod' "I of a modern 

m.ipir disease cfTeis a, textbook picture of the 
interplay nf heredity and environ mem as sc on 
at the moleeubr level (:mti;;ui*Mitii>ody, bili- 
rubin), the cellular level (KRG*s and lyi jpho- 
c\tes), the organ level (uUms. plxceiila and 
liver), and the total body-mind cmnbin dio;i 
(child's IQ and bchavitr and parents* psyche). 
Can this complexity help explain why the dis- 
ease, first desu ihed m (l,o mule in .. 

not recognized in tl.o human till 17fJI— hv: a 
genetic mechanism icpojlcd in io !- -r Vl ,.. i 
the nmle-biil not in man until IfJo'J 1 C n: it 
possibly explain why I .ovine's IS!' 5 ubsom.i’ion 
ii'gai ding Al'O incompatibility pro** 'turn 1 v.ms 
leally not cxpl^T d fei almost two dei.ali s .ml 
then by a htiili ify gineliiist who allege* h was 
his w ife wal iog him in the middle ( f thi night 
who ga\ e him the idea of ln.it.mnl rijnjly by 
using nid;-]ih m-ium? 1 Is this .d] si. t. ply due to 
chance? Tlie “p* v.thic borders on /me Radar 
it r\< inplif>s the cumh.n.ih':i;i of the light f-uts 
in the light mind at the light liin-'-ur tlie 
,f .«h lu" p!u m in m non! 

Rr-.d [].. o p.-atie 1 ► i , i *. 1 1 ( m T n g« i s * ds ’ p ■ t ■ 1 sub* 
lie hints or pnus omldnuii to lbs limy- 



arc, in fact, relatively uiiumi.lI. IY/h.ips d<mm 
rare would be a better term. Fvcn r..rer is a 
tlumipeutie maneuver whose mkuss r.itr bor- 
ders tin Pb pejeuil. And almost unique is »he 
foe- that availability of Rh immune globulin 
occuncd almost simidl.inei udy ihnmghnut the 
country when government legtilalimos permitted 
ils n lease. *1 he bine was right to lake .id\\iul, ge 
of this combinatioa of eunt!J-for an entirely 
different purpose. The Cjvicstion; “13Id tluce 

n.o.illis .suffice fur knowledge of this bu\ik- 
througb to diffuse out to the practicing physi- 
cian? If so, lir»w did it get there? And rnoie 
importantly, did it induce any change in his 
behavior? 

Through the conperaliun of both the King 
County Central blond Ih.nk and tlie Snohomish 
County ;31 o:k 1 hank Association we were pro- 
vided widi a list of physicians, who had dcliN'ercfl 
Rh r.cgrlis'c paiients (pnfenli.il recipients of Rh 
immune globulin) dining the thue naiuths im- 
mediately following ils relearn. The King Couniy 
Cf'ntr.il blond Rank lnd dj.vcU'd a Idler of ar,- 
nonncen.Mit to csa iv physician in the ctjon'y just 
prior to the product's rvle.e e. Its DIjecN r, Ov-m is 
M. Dnimbne, and ils Head of Irmn'.inogcndiis, 
FJohc R. Gilrh'll, hid ah.- di»CLlcd a letter to 
tins J<imnaV as well as to the bulletin of the 
King County Vrdmd -Soiit lv. ))r. Cilddt had 
miner reinfoitvu thess* cffurls with a pel son. 'd 
'sdspl.ore call !o lire physician <1 tvery patient 
with a premia! Rh negative blond typing. Tlie 
bUohaim’sli Hlnnd Rank and its Oircilnr, CLvlon 
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play ;it the spt ing uhxVti ical-gynccoh.gic.d 
meetings in Chiengn .mcl followed with an in* 
formative pamphlet to .ill olMefricianx and 
generalists delivi-j irg babies .it the time of 
Mu jG A. M's rrlr.c r, But, ditl the ine>sagc get 
through? Ami it so, how? 

In the King Comity Wood Hank .sample, 102 
potential recipients i f Hh immune globulin were 
identified during this interval. Of these, 110 
had received it. Tor an additional 33 it v.'as 
not administered for reasons considered valid 
However, 19 failed to r< cciw Rh immune globu- 
lin. At the moment, throe is no reason detectable 
in the Blood Bank records for those 19. In sev- 
eral instances patio t refusal is kno » to have 
occurred because o: cost. The recent, still fmtlaT 
reduction in cost ought to eliminate refusal. 
Regard lesq nt least SS‘ percent of those women 
at risk had ccciwd the inclBaied treatment and 
within the first tnreo nvmdiS of the in' reduction 
of die technique. This raises the question, 
Where is the "application gap" to which so 
many allude? 

Remembering our focus was primarily on the 
use, v. c selected a stratified sample of 21 physi- 
cians— 14 obstetricians mid 10 genera! practi- 
tioners— from the listings. One of us (IIP) then 
telephoned each, and after an explan; to;y intro- 
duction posed que Minns about each of the three 
points noted below. Uniformly, he was accorded 
full cooperation by each of the. practitioners. 
In response to the question, Cm you recall 
where you fh si came in contact with information 
about HhoO.\M?, 12 of the 14 obstetwei ins 
responded via medical journal read ng; only 3 
of the 19 general praelBionors pointed to journals 
with 7 citing verbal encounters cither with col- 
league;. consultants oi detail men. This supports 
Mcmvl ' c onduuons about tl o cnntr utin * mceli* 
anisms for the diffusion of medc.d i.'iforr.iat >*n 
info I In ■ pr.utieiiv; comm mityg a4.ed the 

question. What convinced von to use RhuOAM?, 
a . , ; u th r c-h. te.'riei ms pointed large!)' to the 

\Vi , i:N , ; I <'\\V, \*!v gv'.V.d y* !o '•'.'deal 

coidia '’ji.Jions. 

Dining dm telephone ceinv:xnt?\*n sevud in- 
dued questions were posed aimed at detecting 
t?s* ■ physiiaVs compudv rrien rf the id.yxidogx 
irti\hmi\uv! involved, i e., !Y> you p!,m to \xe 
l\l : GAM for all your Rh negatin' popo-mt pa* 
ti. jits? Why- or why «$»>!? Tvvi rdy two of the 21 
jvqvmdoitf > Wire teiiMfl.uhly previa* in tin ir 
;,ii'Au i s and, u<- mB in d, ui.it*' in lla if < cm- 



prehension and application of the sdime* 
knowledge tmolvcd. Apparently the rornepts 
involved, white new and diffrunl for the pi'adi- 
tioiirrs— although they had been pustulated in 
lQOT— were absorbed with remarkably little 
effort. 

Obviously thru, the appropriate knowledge 
); id diffused out to the medical community and 
had brought about appropriate behavior change. 
Somehow continuing education had taken place 
and its goal of improved patient earn had been 
realized. Possibly the saturation approach of 
wide spread factual dissemination or emotional 
appeal i: the only successful one. If so, it pre- 
sages an impossible situation for the future-* in 
considering the impact of the so-called "knowl- 
edge explosion" :Jso known as “publication 
inflation." Today, there is far too much to read, 
far too many meetings to attend, far too many 
„ TV broadcasts to watch or radio program; t*> 
listen to. bach relevant advance— many ad- 
niUcdly not 99.9 percent effective— cannot bo 
expected to bring about physician behavior 
change simply by random diffusion or Brownian 
movement. 

Consider this ns a possible reason for par- 
ticipating in attempts to analyse objectively a 
inuUiiude of your ccnlmirng education efforts— 
to see if you couldn't do it bluer. Insist mi the 
chance* to express your opinion on the>c matters, 

• and in such a fashion that you won’t be ignored. 
Demand evidence of effectiveness of the edu- 
cational efforts. You have already proven that 
you work; now the burden of proof is with the 
cducatiopnl effort. n 

Hr.i-h P, Fc,x, B.A. 

Will iu in O. Robert SO' i, iU.D. 
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Sock It To Us 



J ^ven the must skcplic.il of skeptics agrees 
-J that in order to stay aheenst of progress, the 
modern | bysidnii must constantly update him- 
self— and at mi ever increasing pace. White a 
few may not sense the need, all evidc. -jour- 
nal subscription conferences held, courses at- 
tended, etc.— confirms that most ?re, in fact, at- 
tempting to do so. Conceivably, we’re not al- 
ways successful— but some recent studies sug- 
gest we may be f, r more so than suspected. 
Consultations, local staff conferences, and med- 
ical journals make up the backbone of the meth- 
od. My guess is they will continue to do so. 
But modern communication methods and trans- 
portation systems expand the possibilities of 
tailoring teaching efforts to individual learning 
needs. 

Admittedly, each of us differs as to which 
methods of learning appeal most-or which one 
works best. T'me of us arc verbalists (or 5s it 
’Verbal! zers”)j others, readers. Some of us are 
“seers"; others arc "listeners." Some Icam best 
from doing; some pci Imps from observing. The 
point is that for none of us is one technique of 
learning in and of itself cither applicable or 
sufficient *.o produce maximum results; in fact, 
a given method may be most efficient for some 
but least efficient for others, lienee we pi ay the 
field— and the field is changing! 

Perceiving this variation among tlffir to* turn- 
ers, primary schools, high schools, colleges, and 
imivcrsi' it s arc offering a buffet, a sinorgasboid, 
a supcTimuket-appcoarh via modem technology, 
Books, lectures, films, audio-tapes, TV, etc. — 
ami various peumilatiuns and combinations 
thereof— the works arc made available. One 
eventual nim is to haw the learner himself 
take responsibility fur picking and chunking the 
must appropriate modality for him. 

Recognizing both the increase in dcsiinhiv in- 
formation (despite r.n obvious "publication in- 
Halion 1 * there is still an "information explosion") 
and the adv.iiit.igc5 of letting the learner pick 
nmi choose which method works best fur him, 
five recent development! in du* Pacific Norlh- 
ve * warrant often tom 



L Severn! years ago a pilot medical television 
series originated in Portland; while evaluation of 
these initial efforts proved somewhat disappoint- 
ing, it’s upwaub and onwards. Aider much local 
and regional planning and local production, open 
channel medical educational TV lias now also 
premiered in Washington. The first pilot offer- 
ing produced under the aegis of the Washing- 
ton/ Alaska Regional Medical Program and fea- 
turing Virginia Mason Clinic’s Dave Fryer and 
John Tytus— “Bedside Diagnosis of Stroke"— lias 
now gone out over the airwaves. Other pro- 
grams arc due to follow. Meanwhile, at the 
local hospital level, tl*j Network for Continuing 
Medical Education (NOME) has installed video 
tape machines for local viewing of any avail- 
able TV tape. In other words, tele vision is on 
tire scene— and more is to come. 

2. In conjunction with the already five-year- 
old California "University Medical Network," 
("Medical Radio Conferences," Tuesday, }2:30) 
FM radio broadcasts, which allow simultaneous 
telephone input from any point in California, 
Oregon, Washington, or parts of Idaho and 
Nevada, now reach all of these states. More than 
22 stations arc involved. Modern technology thus 
permits to and fro dialogue and all of its ad- 
vantages. IIenc<\ in the taxonomy of learning, 
this system overcomes the limitation of simple 
information dispersal. 

T Also, using techniques developed cbowncrc, 
the pby.dci.ms of southeast Alaska— geographi- 
cally so isolated one from another- are now 
joining together via the telephone-conference* 
call-system. Under this arrangement, these phy- 
sicians located at distant points are now able to 
join in discussions among themselves or originat- 
ing anywhere from the "lower •IS" 

4, As noted already in these pugni (mhuii- 
wisi mijvcini:, August lfifiS) a P.r : fic Nmth* 
west Regional Health Sciences Libia iy is now 
being implemented making acce ss to all recorded 
medical information available to any phydeian. 
The concept uhme is enormous. 

5. Finally, recognizing the extent of future 
possibilities practicing physicians, cumwmniU 



hospitals, medical schools, regional medical 
pro groins, state agencies, ami vrhiuhtiy ng'ii- 
ci cs, etc., ,ire working together to capitalize nn 
this technology. For example, in the SYptendier- 
Octnhii* \<m issue nt Oregon Medical Asso- 
ciation’s CmUinuiuiz . lcd : cal Education AVvr c, 
Spence Mcighon reports on the success of a con- 
ference of continuing education-ricks from Ore- 
gon, Idaho, Washington and Hritish Columbia. 
Washington has just witnessed the creation of a 
fornm—a Continuing Medical Fducalinn Coun- 
cil- developed by WSMA, the Washington Med- 
ical Education caul Research Foundation, (he 
Washington/ Alaska Regional Medical Program 
and the School of Medicine— lo zcio in on the 
issues— and bow to exploit technology. Logie de- 
mands that sve must work together and elicit 
local involvement— and, for maximum efficiency, 
develop rot simply a medical but health sci- 
ences- wide continuing education network. 

Lest this seem so much fluff— it is not! In 
Britain, for example, a scant ten years ago s;wv 
less than 5 percent of physicians attend any for- 
mal continuing education course; last year, 
more than 56 percent did so. Already the Oregon 
Medical Association has established, as a require- 
ment for continued membership, evidence of up- 
dating efforts on the part of its members. Martin 
Cummings, Director of the National Library of 
Medicine, is talking of a national biomedical 



network, an integral component of which is a 
.satrlliU’-diiretcu, enW, medic. d television chan- 



nel available to each and every physician over 
the entire continent. People are thinking big— 
they have to. 

The point then is not whether technology >\ 
here to stay; it is, Technocrats abound and will 
be multiplying like rabbits. The issue is which 
controls what— and how soon. Who will his 
master; who, slave? Familiarity with the tech- 
nology may decide the outcome, Thus, a plea 
for each of you to whet your nppetiV by wet- 
ting your feet. Kxpand your her irons; try tape 
recorders, watch medical television, listen to 
medical radio, participate in telephone confer- 
ences— and reach I.eavn about the alternatives. 
Form opinions, and then after having done so, 
continue to cooperate with evaluation efforts, 
Threatening or non-threatening— the question- 
naires aim at the medium, not at yon personally. 
These can help us as individuals as well as 
groups, Decide not simply which medium is 
most popular, but which one works best- and 
for how much. 



Who knows but the year 2000 may see a "per- 
sonalized maximal learning profiled developed 

• for each medical student— and carried forth with 
him as he ventures into practice. It’s our job 
to set the stage for the best of the "brave New 

• World F » 
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Basic Science Problems 




r Ohe year just past was a year of questions, 
JL almost ns much ns it was a year of events. 
Few institutions and feu* ideas have escaped 
scrutiny. .V.d the way the rpicstions arc 
answered will have an effect for a long liiim to 
con e. The idea of n .state di m inding that 
pruclitionn < of the healing avis haw some 
knowledge <F basic sciences has been no ex- 
ception. Usefulness of the basic science exami- 
nation has come undo' serious question and the 



iii'isweis will have their influence on the quality 
of cate provided for citizens for a long linn- 
to come. 

The basic science examination was not de- 
veloped ns a device to estimate profuieiuy It 
pi .ict ice. It was designed to do (lie only tiling 
a piofcs5ion.il liccnring hue con do~r>t nblkh a 
minimum xtnmlutd. It is a screen tlnnugh which 
tho qualified can pa*q regardless of the degree 
of their proficiency, but through which too un- 



PUBLISH, PEkTSH — OK POOP OUT 



Piqued by curiosity, I r:cently updated a study begun aono ton 
years ago. The object: to look at our pediatric journals, to see 

what they are saying, who is saying it, how long they take to say 
it, etc. The subject: Pediatrics , The Journal of Pediatri cs, and 

The Airier le a n Jour nal of Pis oases o f Ch ildren* (at inception, Clinical 
Pediatrics was not yet on the scene)* The method: SimpJ.y cot up 

criteria, count and record data by specif ic journal; then compute 
avoragos, porcontage3, and do on. The years covered: *58, ’59, and 

T G0 versus 1 67 and *63. Nothing too fancy~-Gava for application to 
the recent set of Foinstoin f G memorable categorization cn M Non -human- 
non-disoas papers--ono other way of grappling with tho issue of 
relevance of journal content to problor .3 of practice. 

Each of these three journals provides nearly 1,000 pages of 
textual material every six months. Figuring 700 to 1,000 words por 
page, the average reader proceeding at 3C0 words per minute for 
readlng--not necessai-'ily for "studying" or "understanding"--wculd 
take approximately 160 hours every six months or about six hours a 
veok to consume their respective contents. Even then, he would 
have missod almost G5 per cent of tho pediatric articles abntracte i 
by the Yearbook of Pediatrics --and purported to be important. And, 
worn he to add then, he would or.conj ass only articles immediately 
related to the field. Obviously, the "crunch* i3 here, tho squeeze 
is on-~ond it hay been for a long tine. Some available options: 
read fas tor, become more solectivo, or throw in the sponge. Not 
particularly attractive alternatives; onl> the second car be ccn- 
^ s Mured accent able and even that holds reservations. 

ERLC < \ , 
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Rest rr.surcd, however * that authors have yet to throw In tho sponge', 

As do Solla Price confirms, the infernntieu explosion as ho calls 1 c — or 
is it ctlirip.' y a "publicut: on in Cation" --continues so that output of papers 
and journals doubles about every IS ycar3.^^ While tho rate remains rela- 
tively coijstant--anil has for mere than two centuries !--thu absolute numbers 
of papers continues to burgeon ^aoriotricully, And, no do tho number of 
authors. Of tho nevo than 1,S00 pediatric articles published in the initial 
interval, some 30 per cent had a single author; only 15 per cent "required" 
four or more. Over tho ensuing decade , u changes hem occurred; no w only 25 
per cent of theso pediatric pa porn are silked by an individual} multiple 
authorship- -four or noro- -accounts for 21 per cent; and over tJv; pact five 
years In our Pediatrics Department at the University of Washington , multiple 
authorship accor ts foi T 24 per cent of all papers produced. One wonders why, 
Considerably more drvMruitic--and conjuring up all kind3 of additional 
speculation--is the fact whereas '50-* 50- 'GO saw some 74 per cl at of articles 
spawned in tho medical schools, ’67-* 03 saw more than B7 per cent originate 
there. And of the 13 per cent that did not, a significant proportion stemmed 
from the ar.TJod services-~ano the NIK! By those criteria, practicing pedia- 
tricians arc the "silent majority." 

Conclusion*. Has the time- squeeze for reading lead to a retreat fron 
writing? Possibly, but I doubt it. More plausible possibilities are ob- 
vious* Is it that the two worlds of Tnedicino--acadcu;ta and pruct icu--liko 

(?) 

Snow’s two cultures, nro b imply moving further apart. Practicing \ .jyjiciar. 
apparently arc not recording tholr observations or, if they aru, they are 
having their efforts at writing rejected by Pdltoriai Ecarda--a seemingly 
unlikely occur r ence . Could it be that virtually cvoi y pediatrician a?id bln 
brother is ablo to cecuxe a faculty appointment so that tho "University 
Q V? 1 thou t walls" Is in reality a fact of life? Here thin option correct, it 

4 ! i 
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would portend real excitement for the future* Should It bo another* I.c., 
that of "soparut Inn however, it could spell doom--and not simply fox' 
pediatricians nu<l the profession, bet also their patients. Careful obser- 
vation^ critical analysing and concise, clear recording of clinical experiences, 
is a sine qiui non of the practitioner who p> opes os to continue in the forefia.iit 
of medicine. Although "publish or peri six" may bo the ccoffcd-at motto of 
a cn Corns , it in extremely relevant to Mrs fx'ane of mind. And, .it is perhaps 
oven a more relevant concept for the pxvictitlonor unions he is to ' poop out." 

The process I vs o If , nt: Voodford has emphasised , can be a re vitalising 

( 3 ) 

experience when such aro clearly needed by all of uu. Thus, a plou to 
practitioners for prophylaxis in practice j prepare a paper for publication--* 
promptly ! 
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Do you fool that wi'h nil that's 
going on in the world, you're getting 
birther and further behind the pro- 
fessional times? Evidently v.i* all do, 
though we may wondi r whether it 
really makes a difference or just how 
much. Take heart. Price reminds us 
that our j.rofi ssionnl forebeaicrs ob- 
kcted to tlie same types or pressure; 
for 2 <k> years they also fe’.t t ht> crunch 
of being suniewbat less than lip to 
date. 

In a Inin minus vein, Kelly West ut- 
terly demolishes prohs-donal confi- 
dence in the value of ex; -I ini: knowl- 
edge with the following hit if specious 
reasoning. Of np plica Me medical 
knowledge that will be available in 
1P“0, only half is actually known to- 
day. Of that half, at her.!, 20 per cenl 
is taught hi medical .school. Of ih.it, a 
Mnall part is in error leaving 10 per 
(xnt of which, at most 10 per cent is 
learned, of which conceivably S per 
cent may he retained. <SiH , cinli/.ation 
reduces still fur die r that which is able 
to l»o i 1 st < 1 hy the inch vidua) and some? 
knowledge becomes obsolete. Thus, 
only a h< nt one. per cent of the factual 
information today's mi '<Hc.il student 
..il! tie ush'g in IPSO will have been 
learned b medical school. The rest 
in. ist be gleaned ebrndvre! 

While all of \\< , including West him- 
self, might doubt Hu* pio-dsion <jf Ids 
percentages. none w j 1 1 1 question the 
implication 111 . i i continued learning is 
imperative. To encourage ,ui< h is the 
primary mission of dark bein' s Divi- 
sion of (kmlinuitig Kdi lent ion at the 
Sellout of Medicine mid (ompurah’.e 
units of vuioiw proVs.'lonnl orgaTibu- 
tbns oral Im-pital staffs throughout 
till’ st.de. At :he hkhihiiI, l".iwe\e-, 
irifdi ih-Jy all toa Jitde is known 
about how tlf utors continue b* learn, 
what day rerd to learn, wlial teat h- 
>ng ti i lifiiou .- 1 ,vv bopi . ffei live arid 



efficient, what information is relevant, 
at l what should be retrievable. 

To help look into these questions, 
• an Office of Keiwarch in Medical Kdu- 
eation has recently been activate d at 
the School of Medicine. Under the 
direction of (diaries \V. Dohner, 
Phi)., whose background mid training 
are in die fields of education and 
whose? faculty appointment is in the 
College of Education, one of the first 
efforts will bo directed towards con- 
tinuing medical education. 

Springing up in the post-World 
War II period at the University of 
Ihiffalo, a group of semiriissident med- 
ical faculty members joined with their 
College of Education *o look at what 
they were doing in P :m king to see if 
there might be a better way. Tod **\ 
disciples from die original group 
spread throughout die country j.. .y- 
; .ig n large part in the push which has 
undergraduate curricula in a perpetual 
sf.de of revision at virtually every 
mod’ cal school tlie nation over. (\hnm- 
pioned by (Sourge M M Ior, now at 
1 llinois, "resoarchon-mc-dical-edncia- 
lionists" are attempting to bring some 
elements of the scientific method to 
analysis of medical education. At the 
undergrad late level, <•• ange has 1>cen 
phenomenal; in gradun o training, few- 
er changes have inclined; in coni inn- 
ing education, much remains to be 
done. 

Obviously, the same cdocatUm d 
principles hold fur all time groups, 
Thus the active participation hy lea in- 
tis dieiuseb.es in theii educational 
efforts is c-sm-ii! i;d for optima] learn- 
ing, Vor example, in die end it is only 
tb 1 learner v.lm (nil opine whether n 
giwn bit of information is rrievuid, 
whether a given attitedimd th.imv 
imy bi’ di'-iiebb* or v.hi thi r a :-pe- 
ti f ii * in, mo d -kill nngh! piove In ! pful. 
And. omv g.'i.ds r .r nbjn ti\ca ;ue : pur- 



ified. the sliccoss of an t durational 
program con only be measured in 
terms of the learner's behavior or 
the effects of his behavior or his pa- 
tients — unless one remains naively 
satisfied at equating “effort" Vj.vi.thlrs 
with success instead of “effect'* varia- 
bles It is for these reasons that Ibis 
Office of Keseurc'h m Medical Educa- 
tion will be dependent upon (lit* help 
of learners as individuals and as 
groups, for without their cooperation 
improvement of the educational proc- 
ess will fail. 
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Krsrnrrli ill IMtiralion 

(Con tin u t't i f/n m pat:*. * J Oi* 

usn or would like to u-i- rr dual jour- 
ji.'iIm. 1 * r< ■ 1 i n i i i tary finding urtamly 
Mij'jre.-I tin* 1 k *ti:i fide “Joined of Jr- 
cprud aril .»!“ Results" c > : ] j.:. ! > t to he 
jointed immi'di.dely hy a “h-unul of 
Inci-mpleUi HfMilts" and a “Journal 
of I’trmnluic llesult.V- e.icii calling 
clearly ( < j (lie attention of tl r reader 
the address of the ‘Mi.urnil of Ouilt 

Hcli'.-irtiu] t.s. M 

Owr tin* next few v.c hopi 

to he contacting variou-i individuals 
and groups of physic faiw to be[dn to 
work out ways of dot' ■rmininu how 
physicians use these tools of conlimi 
i m g education. Together We will try 
improvising new a ppi o;i< lies and 
measuring success ot failure. 

As never oefure, hreame c*f the 
social and political jm ■smuts as welt 
as the knowledge explosion, the prac- 
tising ] diysii 'inn has a hig '« , ;d<e in tins 
enterprise How can fie nadd the new 
leehno'ogic s to 3iis needs, rutlu*r than 
have himself simply molded \o them? 
Hramstm mm;:. innsFCiJinm, ardent con- 
victions an? ecitluiMiisin pt r s e will not 
suffice in developing answers. Only 
through If i e active eel hd mi rat inn of 
practicing physicians cm Ada he de- 
veloped and distilled to d<wi-.e upli- 
mal al terna l ives to th d ''lifetime 
learning for physicians” can income 
a reality. We believe this Office of 
Research in Medical Education will 
he most helpfu 1 in achieving the goal. 



Aivnrd to U. W. Stall* Memliei 

Ixah rle Laboratories has announced 
flic? soled inn of Dr. Laid Horusleiu of 
the University erf Washington School 
of Medicine as recipient of a Jjedcrle 
Medical l-’aculty Aw a id. 

r rhe award, developed to assist 
young men aspiring tocaieeis in medi- 
cal teaching and reseaicli, pays pail 
of Dr. Homstein’s salaiy for tliree 
years and ii a lodes S1,N'() to he spent 
at I he awardee’s diMiction toward 
fm (lu ring Ids academic' c.tKrr. 

Dr. Hoitisfein, as.-Llniil professor of 
medicine, joined the medical family 
in 1*107 after rive yens as rcsearUi 
investigator with the National Tn-ti- 
t ill es of llrallh al Uelhe-da, Mow- 
lam ]. 



THOSSTENSON’S PHARMACY 



1000 MADISON • CORNER TERRY 



Prescription Specialists 

OPEN SEVEN DA>$ A WEEK 8 30 q.m.- 9:00 Week Days 

!0-6 Sundays 

MAin 3-2707 




FOR OVER 22 YEARS — 

■ Ciown Hill Hospital has been providing individual caie and 
attention for cmclionaJly disturbed, psychotic- and alcoholic 
patients. 

■ Recognized therapies available including somatic (E.C.T.) 

■ Experienced Psychiatric registered nurses on duly 24 hours. 

* Recreational and occupational therapies, 

• MeavJtifiilly landscaped grounds ar.d patio. 

■ 38 patient capacity. 



Located at 9010 13th Avt\ NAV. 
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Little is known about why students choose to apply to particular 
internships. Less i3 known about how faculty counselors advise them 
or the basis of their advice. Although approximately 70 per cent of 
our students are matched by the National Intern and Resident Matching 
Program with their first choice of internship site, some 30 per cent 
must settle for a recond , third or lover choice. And a few are not 
matched. Were predictors of success or failure available, thc v might 
provide guidance in selecting M appropr late" choices. 

A group of ten faculty internship advisors ware identified; five 
vere Internists; five, other specialties. Each then rated independently 
Into five categories all of the internships (site and type) achieved by 
our 1967 and 1968 graduates, using a modified Q-sort technique, Inter- 
judge reliability was .95, implying significant concurrence among tills 
group. No intra-specialty mean difference was found. (As of yet, the 
basis of this concurrence remains obscure; nonetheless, it seems perva- 
sive among this group.) 

The following data were then assembled for the classes of 1967 and 
1968: undergraduate grade point average; MCAT scores; second-year, 

fourth-year, and over-all class rankings; National Board scores; first, 
second, and third .choices of internships; and sites matched. Next, 
using the average ratings of the internships as the dependent variable, 
ve performed a stepwise regression analysis. Results indicate that 
over-all class rank shares the most variance (.21) with the internship 
achieved. Adding the seven other variables raised the variance to 
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,27--nn increment not considered “significant . 11 Perhaps more important 
is tin.* finding that use of this predictor demarcated 58 per cent of the 
students who failed to match with their first choice where failure to 
match was defined as being nbox'e or below 1 SEest. The majority of 
these students are from the upper third of the claoo who aimed "too high" 
none of t he students shot "too low." The results are being cross- 
validated with data from the class of 1969. 

In addition to the potential help this information may provide for 
advising students, we are interested in invest iga ting the basis of the 
agreement found among our raters. And does this agreement exist among 
advisors at other schools and, if so, how? 
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